Hoisted from the Comments: Journal Impact Factors

19 February 2012 at 10:05 am Leave a comment

| Peter Klein |

An old post of Nicolai’s on journal impact factors is still attracting attention. Two recent comments are reproduced here so they don’t get buried in the comments.

Bruce writes:

There is an interesting paper by Joel Baum on this, “Free Riding on Power Laws: Questioning the Validity of the Impact Factor as a Measure of Research Quality in Organization Studies,” Organization 18(4) (2011): 449-66. He does a nice analysis of citations, and shows (what many of us suspected) that citations are highly skewed to a small subset of articles, so the idea of an impact factor which is based on a mean citation rate is erroneous.  He concludes that “Impact Factor has little credibility as a proxy for the quality of eitehr organization studies journals of the articles they publish, resulting in attributions of journal or article quality that are incorrect as much or more than half the time.  The clear implication is that we need to cease our reliances on such non-scientific, quantitative characterisation to evaluate the quality of our work.”

To which Ram Mudambi responds:

This analysis was already done in a paper we wrote in 2005, finally now published in Scientometrics.

We have the further and stronger result that in many years, the top 10 percent of papers in A- journals like Research Policy outperform the top 10 percent of papers in A journals like AMJ.

So it is the paper that matters, NOT the journal in which it was published. Evaluating scholars on the basis of where they have published is pretty meaningless. Some years ago, we had a senior job candidate with EIGHTEEN real “A” publications — it turned out he had only 118 total citations on Google scholar. So his work was pretty trivial, even though it appeared in top journals.

See also the good twin blog for further discussion.

Entry filed under: - Klein -, Management Theory, Methods/Methodology/Theory of Science. Tags: .

New ebooks — Knowledge on the Cheap Zenger, Felin, and Bigelow on “Theories of the Firm-Market Boundary”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Authors

Nicolai J. Foss | home | posts
Peter G. Klein | home | posts
Richard Langlois | home | posts
Lasse B. Lien | home | posts

Guests

Former Guests | posts

Networking

Recent Posts

Categories

Feeds

Our Recent Books

Nicolai J. Foss and Peter G. Klein, Organizing Entrepreneurial Judgment: A New Approach to the Firm (Cambridge University Press, 2012).
Peter G. Klein and Micheal E. Sykuta, eds., The Elgar Companion to Transaction Cost Economics (Edward Elgar, 2010).
Peter G. Klein, The Capitalist and the Entrepreneur: Essays on Organizations and Markets (Mises Institute, 2010).
Richard N. Langlois, The Dynamics of Industrial Capitalism: Schumpeter, Chandler, and the New Economy (Routledge, 2007).
Nicolai J. Foss, Strategy, Economic Organization, and the Knowledge Economy: The Coordination of Firms and Resources (Oxford University Press, 2005).
Raghu Garud, Arun Kumaraswamy, and Richard N. Langlois, eds., Managing in the Modular Age: Architectures, Networks and Organizations (Blackwell, 2003).
Nicolai J. Foss and Peter G. Klein, eds., Entrepreneurship and the Firm: Austrian Perspectives on Economic Organization (Elgar, 2002).
Nicolai J. Foss and Volker Mahnke, eds., Competence, Governance, and Entrepreneurship: Advances in Economic Strategy Research (Oxford, 2000).
Nicolai J. Foss and Paul L. Robertson, eds., Resources, Technology, and Strategy: Explorations in the Resource-based Perspective (Routledge, 2000).

Follow

Get every new post delivered to your Inbox.

Join 241 other followers