Do You Really Think That Hurricane Maria Killed 4,645 People In Puerto Rico?

A couple is seen from the distance sitting in their home in El Negro community a day after the impact of Maria, a Category 5 hurricane crossed the island, in Yabucoa, Puerto Rico, Thursday, September 21, 2017. (AP Photo/Carlos Giusti)

Last week, the New England Journal of Medicine published a report that got a lot of attention in a lot of places. It is titled, Mortality in Puerto Rico after Hurricane Maria:

From the survey data, we estimated a mortality rate of 14.3 deaths (95% confidence interval [CI], 9.8 to 18.9) per 1000 persons from September 20 through December 31, 2017. This rate yielded a total of 4645 excess deaths during this period (95% CI, 793 to 8498), equivalent to a 62% increase in the mortality rate as compared with the same period in 2016. However, this number is likely to be an underestimate because of survivor bias. The mortality rate remained high through the end of December 2017, and one third of the deaths were attributed to delayed or interrupted health care. Hurricane-related migration was substantial.

Once I gave a briefing to a two-star and was told that my conclusion was “incredible.” Not, he said, “incredible, wow. But IN-credible, bullsh**.” That’s what is striking about the number. The official total is 64. Fine, no sane person believes that is the case. But to get to 4,645 you have to address the problem of where in the hell are the bodies and why does anyone think the blindingly incompetent government of Puerto Rico–a government that a year on still hasn’t restored electrical service to the island–had the skills, the resources, or even the ambition to hide the carnage when, politically, a big body count would be a monetary gold mine for them. And where are the grieving families complaining about thousands of missing people?

The Washington Post’s Fact Checker took valuable moments away from attacking the Trump administration to examine the problem.

The Harvard study took a different approach. The researchers surveyed a random sample of 3,299 households, representing 9,522 people, and obtained answers from about 93 percent. The survey participants identified 38 people who died after the hurricane through December. Of that number, three died directly from medical complications, illness or trauma because of the hurricane, and 12 died because of the interruption of medical services in the aftermath. The other 23 deaths were reported to be unrelated to the storm.

From that base of survey data, the researchers extrapolated to the whole island and came up with a range of excess deaths. That range is 793 to 8,498, with a 95 percent confidence interval. This is a statistical estimate the means that about 95 of 100 samples of the same size and design drawn from the same population would fall with the range noted in the article. The widely reported number of 4,645 is simply the midpoint and is no more or less valid than any other number in the range.

In response to questions, the researchers posted a document that includes this question: Does your study say that 4,645 died? The answer: “No. We provide a 95% confidence interval of 793 to 8,498, and 4,645 falls in the middle of this range.”

Let’s be clear. If your 95% confidence interval covers a range of numbers from some (793) to a sh** load (8,498) you don’t really have an estimate. You have what is known as a “scientific wild-assed guess” aka SWAG, which is nothing more than a “wild-assed guess” to which you’ve assigned numbers. And while the research team provides context separately, the abstract, which is in a pull-quote above, makes no such caveat. It gives the impression that the 4,645 number is a hard, fast number.

I think this is much closer to the truth.

Steven Kopits of Princeton Policy Advisers, a critic of the Harvard study, notes that the available registry data, which still is incomplete, shows 654 excess deaths above the previous year through December. On the face of it, he said, the Harvard number makes little sense because that means 4,000 bodies would be missing – when only 45 were reported as missing as of December.

“With the power outage following the hurricane, residents in ill health and near death died prematurely for lack of access to life-sustaining services like respirators, dialysis and air conditioning,” he said. “These deaths were principally attributable to an extended loss of power. On the other hand, most of these people were under the care of family or professional staff, and their deaths were recorded in near real time. Virtually all those who died in 2017 have been properly accounted for. There are materially no missing bodies.”

He suspects that because the “vast majority of those who died prematurely would have died in a few months even absent the hurricane,” the number of excess deaths should “decrease month by month and probably disappear entirely — from a statistical perspective — within a year.”

Another critic is Donald Berry, professor of biostatistics at the University of Texas MD Anderson Cancer Center in Houston. “The results are statistically weak and nearly useless, at least insofar as number of deaths is concerned,” he said. “Another way of conveying the confidence interval 793 to 8498 is 4645+/-3852. The error is almost as big as the estimate.” He also faulted the researchers for using a different methodology — official deaths for all of Puerto Rico— as a comparison for 2016. “They should have used deaths in 2016 only the same 104 barrios they considered in 2017,” he said.

The team sampled areas calculated to maximize the death estimate and then they pushed out a huge number that was guaranteed to garner headlines. This, by the way, is not a new tactic. During the Iraq War various academics published papers purporting to show extremely high numbers of civilian deaths. Those studies, like this one, are more correctly viewed as political statements than as actual research. The fact that it appears in a high-impact journal like the NEJM simply shows the degree to which academia has decided it would rather be a political player than an honest broker.