Oops

To err is human; to forgive, divine – Alexander Pope

We all make mistakes. Here are a few:oops

  • Take a cab!
    • Researchers found possible traces of bubonic plague on New York City subways. Later, they clarified that there was no strong evidence these organisms were present or could make a person ill.
  • Ebola will wipe out millions!
    • The CDC estimated that 1.4 million people could get the Ebola virus. As it turned out, the estimate was “off” by 98%: 21,000 cases were documented.
  • Your gut has ESP!
    • Psychologists measured a small but real ability for our bodies to predict the future. It helped that the study changed its methods in mid-stream to pick up this effect too.

And my favorite . . .

  • Rabbits: beware of cell phones!
    • Scientists found lowered sperm counts in rabbits whose testicles were near cell phones in standby mode. (Did they give the rabbits little holsters? Offer them unlimited text and data?) This study was later retracted for lack of evidence.

Scientists are human, and even if the computer simulations eliminate human error, someone still has to tell the computer what to do. A multiple linear regression performed by the computer will always be correct, from its coefficients down to its p-values. It still means nothing if the data inputs are flawed.

In the example of the Ebola estimate, the data was not flawed and indeed the modeler stands by his work. The Ebola case is an example of what a person says and what a mathematician hears. Martin Meltzer was asked for a worst case scenario of Ebola cases. That is what he produced – the outermost, highest number of cases that might occur. My guess is that the people asking actually wanted a reasonable-case scenario or most-likely-case scenario. In math, those would be called expected levels, but “expected” would not come to mind when dealing with an aggressive infection.

In the hysteria of the Ebola outbreak, perhaps the audience was ripe for the higher number. This is true not just for infection news, but also for almost any kind of health-related news. Novel, new, and surprising things capture our attention. They also capture the headlines, the Twitter-sphere, and whatever else draws eyeballs and clicks.

The problem is that the real story never gets to the front pages. The retractions, the oops follow-up gets buried in the back sections, or never appears at all. The bubonic plague study has not been retracted, for example. (Hopefully someone told the rabbits they could still put their phones in their front pockets.)

Thus is it that someone can, in all good faith, say that “I read a study that showed . . . insert surprising, absurd or other result here” and he is right. But also wrong at the same time. Perhaps in getting to the truth, a bit of rough-and-tumble is inevitable, given our human condition. And there we will ever rely upon divine forgiveness.

This entry was posted in Strange Healthcare Numbers. Bookmark the permalink.

Comments are closed.