Last weekend, I patiently sat in my tiny downstairs bathroom with my three daughters and a laptop during two sequential tornado alarms. We followed the storm on twitter until it was safe to emerge. My husband didn’t fit in the bathroom, so he kept his eyes and ears ont he storm outside, occasionally complaining about high tornado warning false alarm rates.
My husband and I both have roots in the Midwest and are used to frequent tornado warnings. In Illinois, where sirens sounded during every tornado warning. Before I had kids, I usually ignored the alarms unless it looked or sounded ominous outside. I take shelter these days, at least when I know about the alarms. We don’t have tornado sirens in Virginia, so we often miss them. This makes me uneasy, since there are a lot of tornadoes in Virginia, even though Virginia isn’t in “tornado row.” But TV warnings and social media are usually sufficient for alerting me during warnings.
Tradeoffs.Tornado warning are supposed to be conservative, meaning that the false alarm rate is high in order to ensure a low false negative rate. The false negatives–the alarm not signaling before a tornado strikes–can be deadly. I’m OK with a lot of false alarms so long as I can keep my kids safe. There are social costs to have too many false alarms in order to drive the false negatives to zero. Namely, we will all have in our storm shelter during a perpetual tornado warning. There are social costs with having a reasonable number of tornado warnings. The main issue here is that false alarms are akin to “crying wolf”–tornado warnings are eventually ignored, leading to people being in danger when there really is a tornado.
It turns out that there is some research on this topic.
Error rates. First, I would like to note that there is about a 7% false negative rate. It’s not as low as I would like, but it’s certainly quite low considering the rarity of tornadoes. No system has perfect sensitivity and specificity. Now let’s discuss false alarms. The National Weather Service defines a false alarm ratio (FAR):
FAR = unverified warnings / (verified warnings + unverified warnings)
A paper by Kevin Simmons and Daniel Sutter in the Weather, Climate, and Society (an American Meteorological Society journal) studied areas with lower and higher false alarm rates. They found:
A statistically significant and large false-alarm effect is found: tornadoes that occur in an area with a higher false-alarm ratio kill and injure more people, everything else being constant. The effect is consistent across false-alarm ratios defined over different geographies and time intervals. A one-standard-deviation increase in the false-alarm ratio increases expected fatalities by between 12% and 29% and increases expected injuries by between 14% and 32%. The reduction in the national tornado false-alarm ratio over the period reduced fatalities by 4%–11% and injuries by 4%–13%. The casualty effects of false alarms and warning lead times are approximately equal in magnitude, suggesting that the National Weather Service could not reduce casualties by trading off a higher probability of detection for a higher false-alarm ratio, or vice versa.
The April 27, 2011 tornado outbreak in Alabama cost many lives and resulted in some discourse on the false alarm rate and what people are willing to live with. A meteorologist from Alabama argues for fewer tornado alarms:
I firmly believe apathy and complacency due to a high false alarm ratio over the years led to inaction in many cases that could have cost lives.
The FAR (false alarm ratio) for many NWS offices when it comes to tornado warnings is in the 80-90 percent category. I say this is simply not acceptable. Sure, the POD is excellent (probability of detection), but if most of the warnings are bad, then what good is a high POD?
Punk Rock Operations Research 
July 3rd, 2012 at 4:53 pm
Post 2007, MSU installed an SMS warning system that covers incidents of violence, tornadoes and probably other calamities — so I now get SMS tornado warnings. (It’s the midwest, so we also have sirens, TV news etc.) I don’t react to tornado “watches”; if it’s a “warning”, I check for sightings, likely paths etc. and stay indoors (but don’t automatically head for the bathtub). I’m not sure that better education (or endless reruns of Twister) will offset the “cried wolf” effect, though. Perhaps more alert levels, and more specific geographic targeting, could help? (“If you live in Hooterville, [stay close to home | stay indoors | head for the basement or bathroom | DUCK!!!].”)
July 3rd, 2012 at 5:35 pm
This is really interesting and a great example of applied probability to use in teaching. We don’t get tornadoes here, but we do currently get earthquakes. These come without warning, which has the advantage of no false alarms, but the disadvantage that we have an entire city on edge waiting for the next one.
July 4th, 2012 at 12:00 pm
[…] Source […]
July 4th, 2012 at 12:48 pm
We have a weather radio that sounds a scary alarm when a tornado warning is issued for our area (the radio is programmed with a specific code for our county). Thus, we are aware of any tornado warning for our county. But in 16 years we’ve never had a tornado go through our small town (for which I am very thankful)! So our conditional tornado risk (given a tornado warning) seems to be very low.
So, when we get a tornado warning, we turn on the radio and the TV and try to find out precisely where the worst weather is and where it is going. It is almost always going somewhere else.
Still, I want that weather radio to wake me up just in case.
I did hear during a recent spate of tornado warnings that newer weather radar can identify even smaller tornados (or tornadic activity), so we may have more warnings in the future.
July 5th, 2012 at 8:01 am
[…] To read more including how McLay’s compliance changed with time, the role of sirens, the importance of educating people about what to do when they hear a siren, the chance of dying in a tornado versus being shot in a mass shooting event on one college campus, how social networks can change things, and other interesting details about the issue, click here. […]
July 6th, 2012 at 9:50 am
[…] Laura McLay discusses the optimal false alarm rate for tornado warnings. […]
October 28th, 2012 at 9:54 pm
[…] McLay wrote a very interesting post entitled. “what is the optimal false alarm rate for tornado warnings?” . A high rate of false alarms is likened to the “boy who cried wolf”, to whom nobody […]
May 11th, 2016 at 10:43 pm
Your blog leads this reader to wonder if you are aware that a weather alert radio will inform you of warnings even when you are “unplugged” and that sirens are not designed to alert people who are indoors.