WEIRD Science

27 November 2010 at 6:12 am Nicolai Foss 1 comment

| Nicolai Foss |

It seems to be rather generally accepted that the Gold Standard of empirically-based science is the randomized experiment. Mosts economists and management scholars subscribe to this view, although its critics include notables like James Heckman (here). Arguably, the greatest badge of honor that one can aspire to nowadays as an economist (let’s forget about management scholars here ;-)) is to publish an experimentally-based paper in Nature or Science. However, one thing is the method of randomized experiments per se; quite another thing is the actual design of such experiments in social science and psychology.

In a recent paper, “The Weirdest People in the World,” Joseph Henrich, Steven Heine, and Ara Norenzayan point out that most designs involve samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies, in practice often first-year students.

Of course, any serious experimental paper should be forthcoming about potential problems of external and ecological validity. The problem is certainly not neglected; in fact, some journals ban papers based on experiments involving students. However, the point of the Henrich et al. paper is to document how massive the problem really is in terms of the extremely widespread use of samples drawn from a total outlier population, namely WEIRD people and the sweeping conclusions drawn from experiments using WEIRD subjects. To establish this they compare to non-WEIRD samples. They end their paper by discussing what may be done in terms of practical research heuristics and research policy with respect to dealing with generalizability.

Here is the journal version of the paper (as well as various interesting comments). And here is the working paper. Warning: The Intro may not be for the faint of heart.

Entry filed under: - Foss -, Methods/Methodology/Theory of Science, Recommended Reading.

My Brush with Obamacare Entrepreneurial Ability as a Latent Variable

1 Comment Add your own

1. Randy | 28 November 2010 at 3:20 pm

Fascinating paper, especially when one reads the accompanying comments. I thought the defense of using undergraduates by the economist Gächter was weak.

My all-too-casual empiricism suggests that among the other behavioral scientists to which Henrich et al allude, the OB and marketing scholars use more undergraduate samples than other fields in western B-schools.