The New York Times web version has a section called Visuals, which looks like an attempt to be relevant in the age of Big Data. This week features Fewer Helmets, More Deaths, which seems so common-sensical that they didn’t put enough effort into the supporting charts. The worst offender is below. For what makes it sloppy, follow the swirl.
Florida motorcycle fatalities
Of course there is a big jump in non-helmet fatalities right after the helmet law is repealed, because of all the riders who wore a helmet only to avoid a ticket. For subsequent years, there is no way to tell if the growth is because helmets are lifesavers (although that’s what transportation officials believe), or because more and more riders choose to forego them. If motorcycling has become more dangerous, we could be seeing an increase in the number of helmeted fatalities, as indeed we do, even if the number of riders with helmets were actually dropping.
All of the imprecision could have been avoided with one simple number: what percentage of riders use helmets. (More precise, if harder to determine, would be what percentage of motorcycle-miles are ridden with helmets.)
Since the chart shows they are about 60 percent of the fatalities, the extent helmet use is below 60 percent can be used to define the relative risk. It’s hard for me to believe that 60 percent of Florida riders go helmetless, and it must have been just as hard for the Times’ editors. The chart, as it stands, is an example of expectation bias, where alternative explanations that don’t fit pre-existing beliefs are ignored.
The story has the analogous chart for Pennsylvania, which is similar except that the most recent data have about equal fatalities for helmet and non-helmet.
The story’s other charts of different series show the same error, but not as blatantly. Motorcycle deaths have become a much larger percentage of vehicle fatalities since the partial repeal of helmet laws. However, the first chart in the sequence shows that to some extent, this is a function of fewer automobile fatalities. Assuming that the drop in automobile fatalities is a function of more air bags and diverse safety features which have no correspondence on a bike, we would have seen at least some increase regardless of helmet laws.
The Times could have looked up the data to see if motorcycle fatalities had gone up more in states with repeal than in states that retained a helmet law, optimally pairing states with similar automobile fatality rates or otherwise controlling for highway type, weather, and other conditions. That would tend to quantify the danger of going without a helmet. Guess they’re still short a data scientist!