Organizations and Markets

Bob Higgs on Peer Review

9 May 2007 at 12:45 am Peter G. Klein 4 comments

| Peter Klein |

Robert Higgs, an economic historian with a distinguished record of publication in mainstream journals, puts peer review in perspective:

Peer review, on which lay people place great weight, varies from important, where the editors and the referees are competent and responsible, to a complete farce, where they are not. As a rule, not surprisingly, the process operates somewhere in the middle, being more than a joke but less than the nearly flawless system of Olympian scrutiny that outsiders imagine it to be. Any journal editor who desires, for whatever reason, to knock down a submission can easily do so by choosing referees he knows full well will knock it down; likewise, he can easily obtain favorable referee reports. As I have always counseled young people whose work was rejected, seemingly on improper or insufficient grounds, the system is a crap shoot. Personal vendettas, ideological conflicts, professional jealousies, methodological disagreements, sheer self-promotion and a great deal of plain incompetence and irresponsibility are no strangers to the scientific world; indeed, that world is rife with these all-too-human attributes. In no event can peer review ensure that research is correct in its procedures or its conclusions. The history of every science is a chronicle of one mistake after another. In some sciences these mistakes are largely weeded out in the course of time; in others they persist for extended periods; and in some sciences, such as economics, actual scientific retrogression may continue for generations under the misguided belief that it is really progress.

This is from a piece on the role of peer review and scientific consensus in the global-warming debate. Notes Higgs: “Science is an odd undertaking: everybody strives to make the next breakthrough, yet when someone does, he is often greeted as if he were carrying the ebola virus. Too many people have too much invested in the reigning ideas; for those people an acknowledgment of their own idea’s bankruptcy is tantamount to an admission that they have wasted their lives.”

Of course, admitting these problems does not by itself suggest an alternative. As discussed here, the challenge is to design an institution that minimizes both Type I error and Type II error, taking into account the costs of each (presumably the costs of the former — missing an important breakthrough — are higher than the costs of the latter, which are largely wasted trees, electrons, and time, though the lower overall signal-to-noise ratio increases the chance that a published breakthrough will be overlooked).

Entry filed under: - Klein -, Methods/Methodology/Theory of Science.