The Peer-Review Fetish

29 September 2010 at 11:24 am 5 comments

| Peter Klein |

I respect peer review as much as the next person and have done my share of publishing in peer-reviewed outlets. But I question the belief, expressed often in academic, media, and policy circles, that “not peer reviewed” means “worthless” and “peer reviewed” means “should be accepted without question.” (A corollary belief is that “funded by a private foundation or company” means “biased” while “funded by a government grant” means “neutral.”) In practice, the distinctions are not nearly so clean.

My thoughts on this were triggered by a revealing statement from Ronald Coase, quoted by Josh Gans and George Shepherd in their study of famous economics papers that were initially rejected, about his limited experience with peer review: “I have never found any difficulty in getting my articles published. I have either published in house journals (e.g. Economica) or the article was written as a result of a request (e.g. for a conference) and publication was assured.” Certainly no one would discount the importance Coase’s 1937 and 1960 papers because they weren’t rigorously peer reviewed. (Can you imagine the inane referee remarks that “The Problem of Social Cost” would have generated?) More generally, consider the Journal of Law and Economics during Coase’s editorship in the 1960s and 1970s — the high-water mark of the JLE‘s influence. Or, for that matter, Public Choice under Gordon Tullock, the JPE under George Stigler, or the Journal of Libertarian Studies under Murray Rothbard. These were edited somewhat unevenly, led by charismatic and strong-willed editors with idiosyncratic tastes, yet have been vastly influential in their respective fields.

Peer review serves a useful function and probably improves the quality of published output, on average. But let’s not make a fetish of it.

Entry filed under: - Klein -, Institutions, Methods/Methodology/Theory of Science, Myths and Realities. Tags: .

Elgar Companion to TCE Oliver Williamson to Win Physics Nobel

5 Comments Add your own

  • 1. Abhi S  |  29 September 2010 at 11:51 am

    Someone may find this study interesting: http://www.springerlink.com/content/g1l56241734kq743/. Moral of the story: peer reviewers are vulnerable to biases. Go figure.

  • 2. srp  |  29 September 2010 at 1:28 pm

    There was a time when the readership of a journal was important. Peer review as a way of keeping things out that few would want to read made sense from the editor’s point of view as long as he could trust/monitor the reviewers. But a strong editorial hand could be just as effective. The peer reviewer was there to help make the journal interesting rather than as a gatekeeper to legitimacy.

    Today, most potential citers of articles in economics have read them before they were officially published and journals serve a kind of certifying and archiving purpose. Few sit down with an issue of a journal and go through it as they once might have. So peer review has shifted to a role for which it is less suited–certifying articles as worthy of being in the official public record. (I ignore here the various abuses that we all know about–I’m focusing on how peer review works now when it is operating in what is considered its legitimate mode.)

  • 3. Thomas  |  29 September 2010 at 1:45 pm

    I don’t think very many people actually think ‘that “not peer reviewed” means “worthless” and “peer reviewed” means “should be accepted without question”.’ But I do think there is an argument for a view like the following:

    If someone has published no peer-reviewed articles then that someone has not shown what he or she is worth to the academy, and if someone publishes 1 or 2 peer-reviewed articles every year then their worth, as academics, is beyond question. That doesn’t settle all the issues of merit, nor even of whether they should be employed at a university (and certainly not what university or in what sort of position). But it does help us to decide what someone is “worth” and what questions can be left on the side.

    My view in a nutshell: Peer review is full of flaws. That one paper has passed peer-review and another has been rejected proves very little about their relative quality. But if a certain topic is never written about in peer-reviewed articles, or a certain researcher never publishes such articles, then we have a good (if still imperfect) indication that the topic or researcher are on the fringes of research.

  • 4. Bo  |  3 October 2010 at 3:12 am

    hmm…I wonder. Most “high-quality” peer-reviewed journals of today seem to be rather incremental and low-risk taking in their nature; quality seems to be equated with methodological rigor and then the unfortunate constant quest for “theoretical contribution”, which in essence is a way for reviewers and editors to reject any paper they do not like – to expect the 1000+ academic articles published every year to all make a theoretical contribution is simply silly.

    Peer-reviewing serves an important role in that it somewhat secures a minimum of “academic standard” (whatever that is) and acts as a way to legitimize the contribution. In some cases it may also serve its role as a way to reduce the individual power, influence and thoughts of the editor, however, given that editors choose the reviewers and choose which reviewer(s) to pay most attention to….its role here is questionable….

  • 5. Oltre la peer review?  |  4 October 2010 at 12:32 pm

    [...] peer review è importante ma non facciamone un feticcio: the-peer-review-fetish   come si riconosce in questo blog tenuto da studiosi internazionali di economia [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Authors

Nicolai J. Foss | home | posts
Peter G. Klein | home | posts
Richard Langlois | home | posts
Lasse B. Lien | home | posts

Guests

Former Guests | posts

Networking

Recent Posts

Categories

Feeds

Our Recent Books

Nicolai J. Foss and Peter G. Klein, Organizing Entrepreneurial Judgment: A New Approach to the Firm (Cambridge University Press, 2012).
Peter G. Klein and Micheal E. Sykuta, eds., The Elgar Companion to Transaction Cost Economics (Edward Elgar, 2010).
Peter G. Klein, The Capitalist and the Entrepreneur: Essays on Organizations and Markets (Mises Institute, 2010).
Richard N. Langlois, The Dynamics of Industrial Capitalism: Schumpeter, Chandler, and the New Economy (Routledge, 2007).
Nicolai J. Foss, Strategy, Economic Organization, and the Knowledge Economy: The Coordination of Firms and Resources (Oxford University Press, 2005).
Raghu Garud, Arun Kumaraswamy, and Richard N. Langlois, eds., Managing in the Modular Age: Architectures, Networks and Organizations (Blackwell, 2003).
Nicolai J. Foss and Peter G. Klein, eds., Entrepreneurship and the Firm: Austrian Perspectives on Economic Organization (Elgar, 2002).
Nicolai J. Foss and Volker Mahnke, eds., Competence, Governance, and Entrepreneurship: Advances in Economic Strategy Research (Oxford, 2000).
Nicolai J. Foss and Paul L. Robertson, eds., Resources, Technology, and Strategy: Explorations in the Resource-based Perspective (Routledge, 2000).

Follow

Get every new post delivered to your Inbox.

Join 241 other followers