More on Prematurity

29 March 2007 at 3:13 pm Peter G. Klein 6 comments

| Peter Klein |

We noted before some work on prematurity, the phenomenon in which scientific discoveries are initially resisted because they lie too far outside the mainstream consensus.

Here is a paper — appropriately enough, not yet published — listing discoveries resisted, and scientific papers rejected, even though their authors would go on to win Nobel prizes for these same discoveries. (HT: Bayesian Heresy.) All the examples are from the hard sciences, but I was reminded of Joshua Gans and George Shepherd’s “How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists” (Journal of Economic Perspectives, Winter, 1994). Akerlof’s “Market for Lemons” was rejected by three journals before the QJE agreed to publish it in 1970. Robert Lucas’s “Expecations and the Neutrality of Money” (1972) was dismissed by the AER as too technical. William Sharpe’s 1964 paper introducing the CAPM model was rejected by the Journal of Finance because of its “preposterous” assumption that investors share common beliefs (a new set of editors subsequently accepted a revised version). There are many other examples.

These stories are interesting, but I’m not sure they tell us much about the journal publication process, or scientific discovery, more generally. After all, there are surely many more examples of Type II error than these examples of Type I error — pick up the current issue of your favorite academic journal if you don’t believe me! Would a different system of peer review, or an alternative sociology of science, produce a better overall result?

Entry filed under: - Klein -, Methods/Methodology/Theory of Science.

See, We Can Study Religion Too The Kaleidic Career

6 Comments Add your own

1. Joshua Gans | 29 March 2007 at 4:37 pm

Actually, economics is somewhat of an outlier in terms of rejections. Far far more than the natural sciences for example.

The issue is whether we should weight Type I and Type II errors equally in science. If a result is true and we are talking about the contribution, then surely we do not want many Type I errors at all. Better to have minor contributions available that breakthrough ones lost to history.
2. Peter Klein | 29 March 2007 at 4:48 pm

Joshua, that is surely true. But what are the ideal weights, I wonder? Type II error isn’t costlless. One cost is the lowering of the overall signal-to-noise ratio, which increases the likelihood that significant contributions, even when published, will get overlooked.
3. CoreEcon » Blog Archive » Nobel rejections | 29 March 2007 at 4:52 pm

[…] than the natural sciences. Well, it seems that the natural sciences are not immune. This paper [HT: Organizations & Markets] documents many cases in the natural sciences. Here is a typical entry: The same Journal of […]
4. Tom Schenk | 30 March 2007 at 12:45 pm

In general, choosing theories in economics is difficult because it harder to show one theory is better than the other. There is evidence that the “softer” sciences have more rejection rates, which makes prematurity even more relevant for economists than physicists these days. I think this can be traced to the quasi-testing of theories in economics. Do we ever really ‘test’ a model?

Also, a good publication discussing the publication process and theory selection in economics, read Canonizing Economic Theory by Chris Mackie. The whole book has a very Kuhnian feel–needless to say, it would be liked by the authors here.
5. Marcin Tustin | 1 April 2007 at 9:20 am

Electronic publishing systems (primarily based on blogs, I think rather than the current e-publishing models), which expose the workings of the submit, peer-review, and publish model would probably help by making all submitted papers, and comments, available, and to the extent that technology allows, searchable. I’m about to write a comment about this on my blog.
6. Marcin Tustin | 1 April 2007 at 9:50 am

My thoughts are here: academic publishing and blogging.