Medicaid supporters have misunderstood this important study

There have been a lot of discussions and debates on the Midicaid eligibility cuts that Congress has just passed and, in particular, what they can mean for the current beneficiaries of Medicaid. A key evidence in this debate was the result of Oregon Health Insurance Experiment (Ohie), a randomized trial, which I helped to direct, by examining the impact of the coverage of adults not insured at low income with Medicaid for one to two years. Although it is always rewarding to see your work used in political deliberations, it is frustrating when the results are misinterpreted.

An important bonding point is the interpretation of so -called “zero results” – estimates of the Medicaid impact that we cannot distinguish statistically without effect. In the case of the ERIA, we have found no evidence of statistically significant impacts of the coverage of Medicaid on mortality or on several physical health measures, such as hypertension, high cholesterol or diabetes.

Unfortunately, people make a common error: they poorly interpret the lack of impact evidence as proof of any impact.

For example, two economists recently wrote a letter to the Wall Street Journal noting that:

“The best evidence on the effects on the health of Medicaid’s expansion come from the health insurance experience of Oregon. The Oria is a randomized controlled trial, or RCT – The Order Stonal for such research.… The East has found no improvement in mortality or any other physical result of the Medicaid expansion. ”

Null results can be extremely precious. They can make us question what we think we know and stimulate innovation. When you make decisions focused on evidence, understanding what does not work is just as critical as understanding what works.

However, the correct interpretation of zero results is essential. The results of the Ohie indicated no statistically significant Medicaid impact on several physical health measures or mortality.

But we cannot say that we have found proof that Medicaid No effect on these results. The difference between no evidence of impact and proof of any impact may seem to be a play on words, but when almost 12 million people may lose health insurance, understanding this difference is crucial.

We must look beyond a simplistic summary to know whether or not there is evidence of a statistically significant effect of Medicaid to consider the extent of the estimated effect and the quantity of uncertainty around him. Each research result is delivered with a range of plausible values ​​around it (an interval of trust) which represents statistical uncertainty on the real effect. If this beach includes zero, we cannot exclude any effect. But we cannot exclude other values ​​in the plausible beach either.

Consider some of the results for the health of the Ohie for which there was no evidence of a statistically significant impact of Medicaid. Some of these “zero results” were sufficiently statistically precise to be informative.

An example of any informational result was the results of the study hypertension. My co-authors and I did not find any impact on the coverage of Medicaid in reducing hypertension, and the results were precise enough to exclude much greater estimates from Medicaid’s ability to reduce hypertension that had been found in preliminary and quasi-experimental studies.

In other words, even the maximum profit possible in our range of plausible values ​​was lower than what previous research had found. Thus, from this “zero result” on hypertension, we learned that the Medicaid effect on the reduction of high blood pressure can be smaller than what was previously thought. (Again, however, this is not proof that Medicaid No Effect on hypertension.) It is a useful addition to discussion.

However, the zero results of the Medicaid impact on uncontrolled diabetes rates (i.e. the high levels of glycated hemoglobin) and on mortality were not informative. This stems from a combination of the relatively small size of the Oregon experience sample (only about 10,000 people have acquired Medicaid coverage) and diabetes (approximately 5%) and mortality (less than 1%) levels in the studied population. The result was a high degree of uncertainty.

For diabetes, the range of plausible impacts for Medicaid included zero, but also included the improvements that could be expected given the estimates of the amount of medication, increased use of diabetes drugs and estimates of the clinical literature on what such an increase in drug would predict the improvements in glycked hemoglobin levels.

We could not exclude either any effect on diabetes or the possibility that Medicaid had the very effect that we expected on the basis of its impact on drugs on diabetes. I would call this type of non -informative result.

The results of the study mortality were also non -informative. They could not exclude the possibility that Medicaid has reduced Or increased mortality of a substantial amount. A subsequent randomized controlled trial, much larger and randomized in which nearly 4 million people have been encouraged to register for health insurance, have revealed that health insurance has a statistically significant impact on mortality reduction between 45 to 64 years. The authors of this study explicitly noted that the results were perfectly consistent with the results of the Oria, due to the wide range of plausible mortality effects that we had estimated.

Randomized evaluations can provide some of the most convincing evidence on the impacts of the program, as pointed out by the authors of the Wall Street Journal letter. But the appropriate use of these evidence to shed light on political debates requires understanding that having a result no one does not necessarily mean that a program has no impact. Researchers and political decision -makers have a duty to represent and use evidence, including zero results, in a responsibility.

Amy Finkelstein is a professor of economics at MIT and co-scientific director of J-Pal North America.