The Orpheus Fallacy

Confusing Correlation with Causality in COVID Interventions

In Orfeu Negro (Black Orpheus, 1959), Orpheus descends into Hades to recover Eurydice.  The great tragedy, of course, is that he succeeds in recovering her body but not her living spirit.  The tragedy is compounded by the fact that Orpheus himself perishes.  He’s confronted by Eurydice’s jealous rival; a confusing scene unfolds; Orpheus ends up falling off a cliff with Eurydice in his arms.

For the very young children of the favelas of Rio di Janeiro, the loss of Orpheus is more than a passing tragedy, for Orpheus had convinced them that it was his playing the guitar in the early morning that would induce the sun to rise.  What to do now that Orpheus was gone, and with the faint glow of the morning sun beginning to illuminate the horizon on the Atlantic, there was not much time to remedy the situation.  But, a few of the young children recovered Orpheus’s guitar, and one of them started to play.  The sun started to project a torrent of photons over the horizon. Right on schedule.  Crisis averted.

The tale is charming, of course, because the young children harmlessly confuse correlation with causation: Orpheus would play the guitar; the sun would rise; playing the guitar caused the sun to rise; all the more excuse to play the guitar.  Less charming, however, is the evidence that entities like the Journal of the American Medical Association (JAMA) have advanced (just yesterday) to substantiate the claim that, by availing ourselves of the COVID mRNA therapies – the “vaccines” – “millions of COVID-19 cases and hospitalizations have been prevented and hundreds of thousands of lives saved”.[1]

One may hear or read this kind of claim in passing: “The vaccines have saved countless lives.”  Even Donald Trump has advanced his version in an interview with Candace Owens, something to the effect of “The vaccines have saved millions.”  The JAMA piece cites as its evidence a “research report” from the Department of Health and Human Services (HHS), “COVID-19 Vaccination Associated with Reductions in COVID-19 Mortality and Morbidity in the United States, and an Approach to Valuing these Benefits,” (December 2021).[2]

The authors of the HHS piece estimate that, as of December 2021, “COVID-19 vaccination was associated with reductions of 25.32 million cases, 1.38 million hospitalizations, and 213,000 deaths.”  These conclusions derive from a very standard type of analysis of “count data” involving log-linear regressions or “Poisson regression models”.  The authors regress county-level counts of hospitalizations, cases, and deaths against the proportion, at the county-level, of the population of people aged 18 and older who had been “fully vaccinated”.  More than that, the authors regress counts of hospitalizations, cases and deaths against weekly lagged values of the vaccination rates, with lags reaching back five weeks.

So, the authors play guitar – they toss lagged values of vaccinations rates into a regression equation – and the sun rises … or, rather, hospitalizations, cases and fatalities eventually decline.  But these things were going to decline with or without interventions.  That’s called Farr’s Law.

In a “Letter to the Registrar-General from William Farr, Esq.,” dated June, 1840, William Farr observed that “Five [people] die weekly of smallpox in the metropolis [London] when the disease is not epidemic; …”  So, smallpox was always out there in the miasma.  But why would smallpox (1) sometimes breakout in epidemics and then (2) recede on its own back its baseline rate of about five fatalities a week?  “Why do the five deaths become 10, 15, 20, 31, 58, 88 weekly, and then progressively fall through the same measured steps?”

Farr’s Law is not so much of a law as say, the Second Law of Thermodynamics, as it is an observation of a regularity.  The Second Law of Thermodynamics is basically a statement of the conservation of energy: If one mass of stuff gets hotter, then another mass of stuff will have gotten colder, and vice-versa.  Farr’s regularity is that epidemics have this way of winding down that looks much like the mirror image of the way they had wound up; something clicks and seemingly exponential increase gives way to something that looks a lot like exponential decay.

The difficulty with Farr’s Law is that it says nothing about how far an outbreak (of smallpox, of flu, of SARS, of COVID) will ramp up before receding.  When does the ramping up click off?  It does tell us something qualitative – that outbreaks follow something of a (possibly skewed) bell-shaped trajectory, but it does not tell us how to calibrate that trajectory.  And that is a major problem when it comes to identifying the effects of interventions: If the trajectories of hospitalizations, cases, and deaths are going to recede with or without interventions, how can we disentangle and identify the effects of a given intervention?

We can’t, and the HHS did not – not with a single-equation regression model that ignores the qualitative structure of disease epidemics.  But, all is not lost.  One might be able to craft studies that could potentially identify the effect of vaccines, and this is just kind of thing that Pfizer itself did in late 2021.

In a study titled “Six Month Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine,”[3]  a host of Pfizer researchers report on their randomized control study involving about 44,000 test subjects.  About half were randomly assigned to a control group that would be jabbed with a placebo (saline solution).  The other half would get the jab. 

The idea here is that the control group should look a lot like the group that got the actual jab.  That’s the bit about “random assignment” at work.  One group should not feature a distinctly different age distribution, because that might skew the results. One group should not be more heavily weighted with people who suffer “co-morbidities,” because that could skew the results.  The only important difference between the groups, by design, should be the fact that one got the jab and one got the placebo.

This Pfizer study is the basis for the claim that the Pfizer jab is “95% effective.”  The idea here is that a few hundred people in the control group became infected with COVID.  Only about 5% as many people in the jabbed group contracted COVID.  These people were thus 95% less likely to become infected than the un-jabbed.

That was the top line result, but buried in the report, and subsequently updated by the Food & Drug Administration (FDA), was the result to the question we should actually care about: Did the vaccine actually save people?  Suppose, for example, the vaccine had induced some number of “adverse effects”.  Suppose the vaccine were to induce the deaths of some number of young people by say, inducing myocarditis?  Should we not factor net performance in to our evaluation of total performance?

The net result, reported by the FDA (and dutifully illuminated by Alex Berenson) is that 17 people in the control group died over the course of the study.  But, a total of 21 people in the jabbed treatment group died.[4]  In other words, the control group performed no worse than the jabbed group.  Indeed, were it the case the jab induced the deaths of some number of entirely healthy young people, then one could credibly claim that the control group had outperformed the jabbed group over the course of the study period.

To recount: HHS strummed a guitar, and the sun rose (hospitalizations, cases, and fatalities fell), but their results mean nothing, and it is not obvious that there is anything HHS could do to remedy their study.  There is a place for single-equation Poisson regression of count data, but epidemics do not constitute that place. Basically, Poisson regression is good for estimating phenomena that are driven by underlying “hazard rates” that are constant over time. A classic study is one from 1898 or so about horse-kicking deaths in the Prussian cavalry. In a bad year as many as four fatalities might obtain. In a good year, no fatalities would obtain. “Best practices” might be in place all along, but in some years the cavalry would simply draw a bad card from the deck. Such a result is consistent with a constant hazard rate.

Farr’s Law blows up the premises underlying the Poisson regression. Underlying hazards pertaining to epidemics are not constant but rather evolve over time. They are dependent on recent history, not independent, not constant, not invariant to time. The HHS had no business applying such a model, because the phenomenon of interest doesn’t fit the underlying assumptions of the model.

It gets worse.  Counts of hospitalizations, cases and fatalities have continued to go up and down again and again in countries around the world despite the fact that some of these countries feature very high rates of vaccination.  (Look at Gibraltar or Israel, for example.)  Moreover, we know now that the vaccines affirmatively do not protect people from infection.  We have nice case studies of fully-vaxxed ship crews experiencing outbreaks.  We have the example of fully-vaxxed cruise participants experiencing outbreaks.  We have examples of double-jabbed and boosted officials getting infected. We have the example of researchers isolated in Antarctica, isolated and quarantined before arrival, double-jabbed, boosted and quarantined again, experiencing an outbreak in Antarctica.

These are inconvenient facts.  Pfizer, meanwhile, established some basis for us being able to claim that the average person in their study would enjoy some protection from COVID infection – for a limited amount of time, at least – but the same results fail to demonstrate that the a person randomly plucked from the study would actually enjoy any net benefit.  They might even be affirmatively harmed.

Excess Mortality by State – Who is winning the horse race?

In this short note I pose a very accessible, alternative measure of performance.  I examine “excess mortality” state-by-state in the United States.  And I pose this idea: One state might feature a population more susceptible to COVID infection.  States like Florida and Arizona, for example, may feature disproportionately high numbers of older retirees.  At the same time, however, the climates and living styles of people in such places may afford them some protection from viral infection.  In any case, the population of any given state and a state’s attributes may influence the trajectory of COVID in that state.  Certain states will naturally feature higher rates of morbidity in a given year, and others, on average, will tend to feature lower morbidity.  But, as coronavirus had become more widely dispersed across the country, did some states tend to perform proportionally better than others?  Florida, for example, may experience higher-than-average morbidity in any given year, and one can only guess (correctly, it turns out) that rates of morbidity increased in 2020 and 2021.  But, did those rates increase (again, proportionally) more than, say, in California?

Examining excess mortality in proportion to baseline mortality amounts to a kind of fixed-effects analysis in that controls for differences across states. For each state I calculate total, all-cause excess mortality in 2020-2021 and pose benchmark mortality for 2020-2021.  I then take the ratio of excess mortality to benchmark mortality.  I pose that ratio as a running measure of performance.

Conceivably, this ratio might be negative in a given state.  It would be negative were excess mortality over 2020-2021 to be negative.  That would indicate that a state was performing better post-advent-of-COVID than pre-COVID.  But, that’s not what the data show.  The data show that all states performed worse post-advent-of-COVID than pre-COVID.

I establish the pre-COVID benchmark by examining total mortality by state for the years 2015-2019.  In 2015 total mortality country-wide discernibly edged up from pre-2015 to a new plateau of about 2.8 million fatalities a year.  The oldest baby boomers may have started retiring in large numbers, and that may have induced a “new normal” of higher mortality.  From 2015-2019, however, total mortality has edged up merely as population has edged up.  It is only after April 2020 that total mortality country-wide starts to exhibit some obvious irregularity.

I graphically exhibit excess mortality for the United States in the following graph.

The heavy blue line indicates total mortality by week, and the thin, wavy black line indicates benchmark mortality.  I come up with that thin wavy line by posing a “non-linear regression” by which I map total mortality over the years 2015 – 2019 against a sine wave and a time trend.  I then project benchmark mortality on to the years 2020 and 2021 (through October 39, 2021).  The difference between actual mortality and benchmark mortality indicates “excess mortality”.  Excess mortality is often actually negative.

The idea here is that total mortality may have closely followed the benchmark but for the fact that COVID discernibly imposed itself starting in March 2020.  One can see that, before 2020, there had been important episodes of excess mortality.  January 2018, for example, seemed to be part of an unusually harsh cold-and-flu season.  Since, April 2020, however, excess mortality has been positive, and, in some weeks, spectacularly so.

Note also the red line.  This line indicates total mortality absent fatalities purportedly impinged by COVID.  Since April 2020 these non-COVID fatalities have contributed a measure of excess mortality in most weeks.  Total mortality seems to have edged up even after one discounts the direct effects of COVID.

I apply the non-linear regression to total mortality in every state, and I get good fits for almost all states.  Delaware is an outlier; the algorithm does not converge on coefficient estimates that make sense.  But the regressions on larger states do yield useful results.  I graphically report results for a selection of states here:

This graph indicates a horse race between the states.  It shows that, by June 2020, New Jersey had, on average, registered excess mortality from the beginning of 2020 of more than 50%.  What that means is that total mortality was running about 50% higher through the first six months of 2020 than would have been expected.  COVID was taking a severe toll.  By December 2020, however, New Jersey had weathered the worst of the toll of coronavirus, and excess mortality for the entire year clocked out at about 25%.  By October 30, 2021, nearly a year later, total excess mortality since the beginning of 2020 in New Jersey yet diminished further to a 2020-2021 average of 18.27%.

The country-wide average by October 30, 2021 was 16.46%.  (See the red line labeled “United States”.)  Florida had been performing better than the country-wide average until it overtook it in about September of last year.  As of October 30, 2021, Florida has clocked in a 17.92%.  But that performance is still better than that of California. As of October 30, 2021, California has clocked in at excess mortality of 18.89%.

These trajectories all have the appearance of slowly converging on the United States average (the red line).  Let’s give it another year, but it is true that, as of October 30, 2021, there is much dispersion across states in this measure of performance.  Hawaii is not pictured here, but it is handily winning the horse race as of October 30, 2021 by clocking in with excess mortality of 4.49%.  Texas is losing the race by clocking in at 24.22%.  So, what is going on in Texas and Hawaii?  Is Hawaii’s relative isolation an important factor?  If so, will it yet see a notable surge in COVID infections? Meanwhile, does Texas’s status as a large border state make its population more susceptible to infection coming across the border?  Or, has the population of Texas been subject to important churn? Is the state of Texas in late 2021 appreciably different from what it was in 2019?

These are tricky questions, and it may very well be the case that projecting excess mortality post-2019 by state is not appropriate for all but the states with the most stable populations. That said, the hypothesis stands: performance by state across most states may converge and have the appearance of converging, the ultimate suggestion being: We can’t distinguish effects of COVID interventions by state.  Ultimately all states may end up performing very similarly no matter what they do.

One can imagine applying this kind of ranking by excess mortality to countries.  I note that, as of October 2020, countries like Czechia and Hungary appeared to be performing very, very well by European standards.  One might have been tempted to suggest that the Czechs and the Hungarians had out-smarted COVID.  But the COVID gods ultimately did not spare these countries.  Within a few months excess mortality in these countries was leading Europe.  So, again, have state interventions yielded a positive result or not?  Or have they even induced poorer performance?

---

[1] See Rubin, Rita, “COVID-19 Vaccines Have Been Available in the US for More Than a Year—What’s Been Learned and What’s Next?, Journal of the American Medical Association, January 11, 2022 posted at https://jamanetwork.com/journals/jama/fullarticle/2788213.

[2] Accessed January 12, 2021 at https://aspe.hhs.gov/sites/default/files/documents/cfae1f63ba6d7eeda21d3fcf2a35c6bc/aspe-ib-association-of-vaccination-select-outcomes.pdf.

[3] https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.supplementary-material

[4] From page 23 of the FDA report: “From Dose 1 through the March 13, 2021 data cutoff date, there were a total of 38 deaths, 21 in the COMIRNATY [vaccine] group and 17 in the placebo group.” https://www.fda.gov/media/151733/download