The Wrong Way to Measure Returns to Public Science Funding

9 September 2012 at 9:49 pm 3 comments

| Peter Klein |

A new Milken Institute report purports to show that “[t]he benefit from every dollar invested by National Institutes of Health (NIH) outweighs the cost by many times. When we consider the economic benefits realized as a result of decrease in mortality and morbidity of all other diseases, the direct and indirect effects (such as increases in work-related productivity) are phenomenal.” There are so many problems with the study I hardly know where to begin. For instance:

1. The authors measure long-term benefit to society as real GDP for the bioscience industries. This is a strange proxy. It is well-known that one of the major impact of public science funding is higher wages for science workers. It is hardly surprising that NIH funding results in higher wages and profits for those in the bioscience industry. Moreover, even if industry activity were the variable of interest, don’t we care about the composition of that activity, not the amount? Which projects were stimulated by NIH funding, and were they the right ones?

2. The results are based on a panel regression of the following equation:

Real GDP for the bioscience industries = f (employment in bioscience industry, labor skill, capital stock, real NIH funding, Industrial R&D in all industries) + state fixed effects + error term.

They interpret the coefficient on NIH funding as the causal effect of NIH funding on bioscience performance. E.g.: “Preliminary results show that the long-term effect of a $1.00 increase in NIH funding will increase the size (output) of the bioscience industry by at least $1.70.” But all the right-hand-side variables are potentially endogenous. For instance, the positive correlation between the dependent variable and NIH funding could reflect winner-picking: the NIH funds projects that are likely to be successful, with or without NIH funding. (The authors briefly mention endogeneity but dismiss it as unimportant.)

This is a version of the basic methodological flaw I attributed to the the political scientists lobbying for NSF money. The issue in question — even assuming the dependent variable is a reasonable measure of social benefit — is what bioscience industry output would have been in the absence of NIH funding. (And, even more important, what would have been the direction of that activity.) Public funding could crowd out private funding, and almost certainly changes the direction of research activity, for good or ill.

3. There are a host of econometric problems, aside from endogeneity — no year fixed effects, no interactions between federal and private funds, the imposition of linear relationships, etc.

If I’m being unfair to the authors, I hope readers will correct me. But this looks to me like another example of special pleading, not careful analysis.

Entry filed under: - Klein -, Innovation, Methods/Methodology/Theory of Science, Myths and Realities, Public Policy / Political Economy.

Conference in Honor of Oliver Williamson Interesting Paper on Entrepreneurship and Growth

3 Comments Add your own

  • 1. Allan Walstad  |  10 September 2012 at 9:31 pm

    Peter, I probably won’t get a chance to read the particular report in question, but I’ve seen enough flawed arguments for government funding of science, and enough debunkings, to doubt that you’re being unfair. Thanks for spraying Roundup in the direction of the latest crop of weeds.

  • 2. Ilir Deebran  |  11 September 2012 at 12:15 pm

    Alex Gheg has a new consumer theory that makes more accurate predictions and allows the measurements of previously hidden thoughts, which means we can measure utility growth. Quantity, quality, variety and convenience in one equation.

  • 3. Interesting blog bits | The Dismal Science  |  13 September 2012 at 2:10 pm

    […] Klein on The Wrong Way to Measure Returns to Public Science Funding A new Milken Institute report purports to show that “[t]he benefit from every dollar invested by […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Nicolai J. Foss | home | posts
Peter G. Klein | home | posts
Richard Langlois | home | posts
Lasse B. Lien | home | posts


Former Guests | posts


Recent Posts



Our Recent Books

Nicolai J. Foss and Peter G. Klein, Organizing Entrepreneurial Judgment: A New Approach to the Firm (Cambridge University Press, 2012).
Peter G. Klein and Micheal E. Sykuta, eds., The Elgar Companion to Transaction Cost Economics (Edward Elgar, 2010).
Peter G. Klein, The Capitalist and the Entrepreneur: Essays on Organizations and Markets (Mises Institute, 2010).
Richard N. Langlois, The Dynamics of Industrial Capitalism: Schumpeter, Chandler, and the New Economy (Routledge, 2007).
Nicolai J. Foss, Strategy, Economic Organization, and the Knowledge Economy: The Coordination of Firms and Resources (Oxford University Press, 2005).
Raghu Garud, Arun Kumaraswamy, and Richard N. Langlois, eds., Managing in the Modular Age: Architectures, Networks and Organizations (Blackwell, 2003).
Nicolai J. Foss and Peter G. Klein, eds., Entrepreneurship and the Firm: Austrian Perspectives on Economic Organization (Elgar, 2002).
Nicolai J. Foss and Volker Mahnke, eds., Competence, Governance, and Entrepreneurship: Advances in Economic Strategy Research (Oxford, 2000).
Nicolai J. Foss and Paul L. Robertson, eds., Resources, Technology, and Strategy: Explorations in the Resource-based Perspective (Routledge, 2000).

%d bloggers like this: