Organizations and Markets

The Wrong Way to Measure Returns to Public Science Funding

9 September 2012 at 9:49 pm Peter G. Klein 3 comments

| Peter Klein |

A new Milken Institute report purports to show that “[t]he benefit from every dollar invested by National Institutes of Health (NIH) outweighs the cost by many times. When we consider the economic benefits realized as a result of decrease in mortality and morbidity of all other diseases, the direct and indirect effects (such as increases in work-related productivity) are phenomenal.” There are so many problems with the study I hardly know where to begin. For instance:

1. The authors measure long-term benefit to society as real GDP for the bioscience industries. This is a strange proxy. It is well-known that one of the major impact of public science funding is higher wages for science workers. It is hardly surprising that NIH funding results in higher wages and profits for those in the bioscience industry. Moreover, even if industry activity were the variable of interest, don’t we care about the composition of that activity, not the amount? Which projects were stimulated by NIH funding, and were they the right ones?

2. The results are based on a panel regression of the following equation:

Real GDP for the bioscience industries = f (employment in bioscience industry, labor skill, capital stock, real NIH funding, Industrial R&D in all industries) + state fixed effects + error term.

They interpret the coefficient on NIH funding as the causal effect of NIH funding on bioscience performance. E.g.: “Preliminary results show that the long-term effect of a $1.00 increase in NIH funding will increase the size (output) of the bioscience industry by at least $1.70.” But all the right-hand-side variables are potentially endogenous. For instance, the positive correlation between the dependent variable and NIH funding could reflect winner-picking: the NIH funds projects that are likely to be successful, with or without NIH funding. (The authors briefly mention endogeneity but dismiss it as unimportant.)

This is a version of the basic methodological flaw I attributed to the the political scientists lobbying for NSF money. The issue in question — even assuming the dependent variable is a reasonable measure of social benefit — is what bioscience industry output would have been in the absence of NIH funding. (And, even more important, what would have been the direction of that activity.) Public funding could crowd out private funding, and almost certainly changes the direction of research activity, for good or ill.

3. There are a host of econometric problems, aside from endogeneity — no year fixed effects, no interactions between federal and private funds, the imposition of linear relationships, etc.

If I’m being unfair to the authors, I hope readers will correct me. But this looks to me like another example of special pleading, not careful analysis.

Entry filed under: - Klein -, Innovation, Methods/Methodology/Theory of Science, Myths and Realities, Public Policy / Political Economy.