More on Journal Rankings
26 June 2007 at 3:49 pm Peter G. Klein 4 comments
| Peter Klein |
The HES (History of Economics Society) listserv is buzzing over ERIH, the European Science Foundation’s ranking of journals in the history and philosophy of science. Writes Deirdre McCloskey, for example:
Among [the] many bad effects [of ranking journals] is to encourage people to rank another person not by reading and considering (a sample of) her work but by counting how many Grade A journals she has contributed to. It takes scientific and scholarly judgment out of the hands of actual readers of the actual work and puts it into the hands of the median voter in a beauty contest. It leads to mediocrity in science, such as the practice of using t tests as the sole criterion of importance in statistical studies. The beauty contest is based on rumor, not reading. When reputation rankings include a dummy journal with a plausible sounding name the respondents claim familiarity with the journal and firmly rank it. Don’t we need to stop this corrupt practice, not encourage it?
Other commentators largely agree. David Colander notes that productivity rankings of economists based on journal articles use proxies (journal articles) that “are only a small portion of economists’ total output (which includes teaching, other research, and service) (I estimate 20%) and that emphasis in one reduces emphasis in the others, so the probability of the rankings carrying through is exceedingly small, even if there is positive correlation with other activities.”
You can read the whole thread here (start with David Teira Serrano’s entry “European Reference Index for the Humanities (ERIH).” See also Leland Yeager’s work discussed here.
Entry filed under: - Klein -, Institutions, Methods/Methodology/Theory of Science.
1.
jonfernquest | 26 June 2007 at 10:51 pm
“…the median voter in a beauty contest. It leads to mediocrity in science”
Often a very restricted and elite beauty contest at a private club.
Teaching and working in Southeast Asia, I despair that I never get to read the articles, even about Southeast Asia, that are published in high-priced prestigious journals. Have to go and read my Pierre Bourdieu again, I guess.
Citations (e.g. Citeseer) is a better measure which also provides an incentive to make articles accessible.
2.
Nicolai Foss | 27 June 2007 at 4:30 am
Now we are just waiting for Omar’s take on this ….
3. Peer Review at Jacob Christensen | 9 July 2007 at 6:53 pm
[…] methods teacher, I would also like to draw attention to the question: What validity and reliability problems are there in the rankings that have become so popular both within the academic community and the […]
4. Peer Review — Jacob Christensen | 3 March 2011 at 6:43 pm
[…] methods teacher, I would also like to draw attention to the question: What validity and reliability problems are there in the rankings that have become so popular both within the academic community and the […]