It’s the Economics that Got Small

23 November 2014 at 2:58 pm 8 comments

| Peter Klein |

Joe Gillis: You’re Norma Desmond. You used to be in silent pictures. You used to be big.

Norma Desmond: I *am* big. It’s the *pictures* that got small.

Sunset Boulevard (1950)

John List gave the keynote address at this weekend’s Southern Economic Association annual meeting. List is a pioneer in the use by economists of field experiments or randomized controlled trials, and his talk summarized some of his recent work and offered some general reflections on the field. It was a good talk, lively and engaging, and the crowd gave him a very enthusiastic response.

List opened and closed his talk with a well-known quote from Paul Samuelson’s textbook (e.g., this version from the 1985 edition, coauthored with William Nordhaus): “Economists . . . cannot perform the controlled experiments of chemists and biologists because they cannot easily control other important factors.” While professing appropriate respect for the achievements of Samuelson and Nordhaus, List shared the quote mainly to ridicule it. The rise of behavioral and experimental economics over the last few decades — in particular, the recent literature on field experiments or RCTs — shows that economists can and do perform experiments. Moreover, List argues, field experiments are even better than using laboratories, or conventional econometric methods with instrumental variables, propensity score matching, differences-in-differences, etc., because random assignment can do the identification. With a large enough sample, and careful experimental design, the researcher can identify causal relationships by comparing the effects of various interventions on treatment and control groups in the field, in a natural setting, not an artificial or simulated one.

While I enjoyed List’s talk, I became increasingly frustrated as it progressed, and found myself — I can’t believe I’m writing these words — defending Samuelson and Nordhaus. Of course, not only neoclassical economists, but nearly all economists, especially the Austrians, have denied explicitly that economics is an experimental science. “History can neither prove nor disprove any general statement in the manner in which the natural sciences accept or reject a hypothesis on the ground of laboratory experiments,” writes Mises (Human Action, p. 31). “Neither experimental verification nor experimental falsification of a general proposition are possible in this field.” The reason, Mises argues, is that history consists of non-repeatable events. “There are in [the social sciences] no such things as experimentally established facts. All experience in this field is, as must be repeated again and again, historical experience, that is, experience of complex phenomena” (Epistemological Problems of Economics, p. 69). To trace out relationships among such complex phenomena requires deductive theory.

Does experimental economics disprove this contention? Not really. List summarized two strands of his own work. The first deals with school achievement. List and his colleagues have partnered with a suburban Chicago school district to perform a series of randomized controlled trials on teacher and student performance. In one set of experiments, teachers were given various monetary incentives if their students improved their scores on standardized tests. The experiments revealed strong evidence for loss aversion: offering teachers year-end cash bonuses if their student improved had little effect on test scores, but giving teacher cash up front, and making them return it at the end of the year if their students did not improve, had a huge effect. Likewise, giving students $20 before a test, with the understanding that they have to give the money back if they don’t do well, leads to large improvements in test scores. Another set of randomized trials showed that responses to charitable fundraising letters are strongly impacted by the structure of the “ask.”

To be sure, this is interesting stuff, and school achievement and fundraising effectiveness are important social problems. But I found myself asking, again and again, where’s the economics? The proposed mechanisms involve a little psychology, and some basic economic intuition along the lines of “people respond to incentives.” But that’s about it. I couldn’t see anything in these design and execution of these experiments that would require a PhD in economics, or sociology, or psychology, or even a basic college economics course. From the perspective of economic theory, the problems seem pretty trivial. I suspect that Samuelson and Nordhaus had in mind the “big questions” of economics and social science: Is capitalism more efficient than socialism? What causes business cycles? Is there a tradeoff between inflation and unemployment? What is the case for free trade? Should we go back to the gold standard? Why do nations to go war? It’s not clear to me how field experiments can shed light on these kinds of problems. Sure, we can use randomized controlled trials to find out why some people prefer red to blue, or what affects their self-reported happiness, or why we eat junk food instead of vegetables. But do you really need to invest 5-7 years getting a PhD in economics to do this sort of work? Is this the most valuable use of the best and brightest in the field?

My guess is that Samuelson and Nordhaus would reply to List: “We are big. It’s the economics that got small.”

See also: Identification versus Importance

Entry filed under: - Klein -, Austrian Economics, History of Economic and Management Thought, Methods/Methodology/Theory of Science.

Bottom-up Approaches to Economic Development PhD Strategy Syllabi

8 Comments Add your own

  • 1. michaelfto  |  23 November 2014 at 4:54 pm

    Maybe there only small questions.

  • 2. stevepostrel  |  25 November 2014 at 5:43 pm

    There’s nothing wrong with economic imperialism–why shouldn’t economists bring whatever skills and models they have to educational issues?–but the homeland needs to be secured as well. Generally, the homeland consists of questions about the functioning of markets and institutions, e.g. Are asset markets prone to bubbles and if so what circumstances make them more severe? Are business cycles caused mostly by gaps aggregate demand gaps between potential and actual output or by (substitute alternative x)? What explains vertical integration or diversification? Under what conditions do oligopolists fight aggressively for market share, or try to develop new products, or collude, etc.?

    Many of these “homeland” questions seem to be inaccessible to the kinds of field experiments List is pushing, but I think lab experiments, theory, and careful observational study do shed light on them. If we want to understand asset bubbles, its hard to see how to set up an experiment using real-world financial markets, even though laboratory markets generate and manipulate them rather easily.

    I can think of some exceptions, though. The recent field experiments in India by Van Reenen et al in providing consulting on basic management practices may be an exception to this claim, insofar as the degree to which firms can be expected to be on the efficiency frontier is a homeland question. And things can be done with real-world auctions and buyer behavior that are important for understanding the micro-fundamentals of markets, demand, etc., that are directly relevant to the “homeland.”

  • 3. Curt Doolitlte  |  27 November 2014 at 12:38 pm

    I think I could at this point reframe Samuelson and Mises both as a mutual failure to understand the problems of general rules and arbitrary precision. Mises’ failure was to create the pseudoscientific argent instead of recognizing operationalism and intuitionism as more mature arguments.

    In this sense, Samuelson is half right in practice but meaningless in theory. While Mises was half right in theory and meaningless in practice.

    It’s actually a fascinating problem – which is why I work on it.

    Mises intuited and Samuelson did not, that morality was non arbitrary and could not be divorced from economic theory.

    None on the other 20th century thinkers could solve that problem either.

    We needed another near century to do it.

    The problem now is the uncomfortable task of reinserting morality into economics – And by that I mean quantitatively.

    Because I am fairly certain it is possible.

  • 4. David R. Henderson  |  29 November 2014 at 9:54 am

    Peter, Excellent post. Typo in the second last sentence.

  • 5. Klein, Peter G.  |  29 November 2014 at 10:11 am

    David, thanks on both counts. Typo fixed!

  • 6. David R. Henderson  |  29 November 2014 at 10:21 am

    You’re welcome. You’re fast! :-)

  • 7. Peter Klein  |  29 November 2014 at 10:29 am

    I want to minimize the number of statements Barkley Rosser can misinterpret. :)

  • 8. Richard O. Hammer  |  6 December 2014 at 3:40 am

    Thank you Peter. This philosophy of science business is very interesting and important, I believe. I’m glad to see that other people can be useful when thinking about thinking.

    I’m not sure I agree with what you’ve said. But I do have a some confidence that I fail to grasp it all.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Nicolai J. Foss | home | posts
Peter G. Klein | home | posts
Richard Langlois | home | posts
Lasse B. Lien | home | posts


Former Guests | posts


Recent Posts



Our Recent Books

Nicolai J. Foss and Peter G. Klein, Organizing Entrepreneurial Judgment: A New Approach to the Firm (Cambridge University Press, 2012).
Peter G. Klein and Micheal E. Sykuta, eds., The Elgar Companion to Transaction Cost Economics (Edward Elgar, 2010).
Peter G. Klein, The Capitalist and the Entrepreneur: Essays on Organizations and Markets (Mises Institute, 2010).
Richard N. Langlois, The Dynamics of Industrial Capitalism: Schumpeter, Chandler, and the New Economy (Routledge, 2007).
Nicolai J. Foss, Strategy, Economic Organization, and the Knowledge Economy: The Coordination of Firms and Resources (Oxford University Press, 2005).
Raghu Garud, Arun Kumaraswamy, and Richard N. Langlois, eds., Managing in the Modular Age: Architectures, Networks and Organizations (Blackwell, 2003).
Nicolai J. Foss and Peter G. Klein, eds., Entrepreneurship and the Firm: Austrian Perspectives on Economic Organization (Elgar, 2002).
Nicolai J. Foss and Volker Mahnke, eds., Competence, Governance, and Entrepreneurship: Advances in Economic Strategy Research (Oxford, 2000).
Nicolai J. Foss and Paul L. Robertson, eds., Resources, Technology, and Strategy: Explorations in the Resource-based Perspective (Routledge, 2000).