Instrumental Variables versus Randomized Controlled Trials

11 September 2007 at 12:03 am Peter G. Klein Leave a comment

| Peter Klein |

Despite the popularity of instrumental-variables estimators some empirical social-science researchers suggest dumping structural models altogether in favor of randomized controlled trials (RCTs), as used in biomedical research. (Like evidence-based management, but with substance.) MIT’s Poverty Action Lab (J-PAL) is the home of this movement. Princeton’s Angus Deaton is intrigued, but reminds us that RCTs are no panacea.

The movement is not modest in its claims, and it has attracted a good deal of acclaim from outside the profession. [J-PAL’s Abhijit] Banerjee has argued that the World Bank should cease to fund any activity (including presumably macro policy advice) that has not been previously subject to evaluation by an appropriate RCT. . . . There is much to be excited about in this program. J-PAL and other experimental researchers have come up with several surprising results that upset previous beliefs. And by replicating similar experiments in different settings they are beginning to create an impressive and valuable body of evidence. As might be expected in the first flush of enthusiasm, there has to date been less attention to some of the problems that have bedeviled RCTs in medicine, such as their limited value to physicians in practice, nor to the extent to which RCTs really do solve the standard problems of econometric analysis. (Indeed, many RCT papers subject their experimental results to various econometric corrections and analyses.) And the jury is still out on whether RCTs are any better than large data sets as substitutes for theory.

Thanks to Marshall Jevons (this one, not this one) for the pointer.