<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
		>
<channel>
	<title>Comments on: Are Reviewers Too Powerful?</title>
	<atom:link href="http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/feed/" rel="self" type="application/rss+xml" />
	<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/</link>
	<description>Economics of organizations, strategy, entrepreneurship, innovation, and more</description>
	<lastBuildDate>Sun, 12 Feb 2012 08:11:55 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
	<item>
		<title>By: Jung-Chin Shen</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-2020</link>
		<dc:creator><![CDATA[Jung-Chin Shen]]></dc:creator>
		<pubDate>Thu, 31 Aug 2006 15:00:19 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-2020</guid>
		<description><![CDATA[In the case of economics journals, two papers seem relevant to this thread. 

(1) The Editors and Authors of Economics Journals: A Case of Institutional Oligopoly? Geoffrey M. Hodgson and Harry Rothman, Economic Journal, 1999.

Abstract: This paper examines data on the institutional backgrounds of editors and authors of the top 30 economics journals, identified by their 1995 citation impact. It is revealed, for example, that 70.8% of the journal editors were located in the United States, and twelve U.S. universities accounted for the location of more than 38.9%. Concerning journal article authors, 65.7% were located in U.S. institutions and twelve U.S. universities accounted for 21.8%. Arguably, the degree of institutional and geographical concentration of editors and authors may be unhealthy for innovative research in economics.

(2) Diversity in Economics: An Analysis of Journal Quality Perceptions, Kostas Axarloglou and Vasilis Theoharakis, Journal of European Economic Association, December 2003, 1(6): 1402-1423. 

Abstract: It is still debatable whether scientific diversity is a virtue or a disadvantage for the develop-ment of a discipline. Nonetheless, diversity among scientists with respect to their journal quality perceptions plays an important role in hiring and promotion decisions. In this article we examine the degree of diversity within economics based on the journal quality perceptions of 2,103 AEA economists worldwide. Specifically, we empirically test for factors that might explain differences in an economist&#039;s journal quality perceptions. These factors include an economist&#039;s geographic origin, school of thought, journal affiliation, field of specialization and research orientation. Indeed, we find that a significant degree of diversity in journal quality perceptions exists between economists that belong in different subgroups. These results might explain the frequent debates in tenure and promotion committees where journal standings are used for the evaluation of a researcher&#039;s output.]]></description>
		<content:encoded><![CDATA[<p>In the case of economics journals, two papers seem relevant to this thread. </p>
<p>(1) The Editors and Authors of Economics Journals: A Case of Institutional Oligopoly? Geoffrey M. Hodgson and Harry Rothman, Economic Journal, 1999.</p>
<p>Abstract: This paper examines data on the institutional backgrounds of editors and authors of the top 30 economics journals, identified by their 1995 citation impact. It is revealed, for example, that 70.8% of the journal editors were located in the United States, and twelve U.S. universities accounted for the location of more than 38.9%. Concerning journal article authors, 65.7% were located in U.S. institutions and twelve U.S. universities accounted for 21.8%. Arguably, the degree of institutional and geographical concentration of editors and authors may be unhealthy for innovative research in economics.</p>
<p>(2) Diversity in Economics: An Analysis of Journal Quality Perceptions, Kostas Axarloglou and Vasilis Theoharakis, Journal of European Economic Association, December 2003, 1(6): 1402-1423. </p>
<p>Abstract: It is still debatable whether scientific diversity is a virtue or a disadvantage for the develop-ment of a discipline. Nonetheless, diversity among scientists with respect to their journal quality perceptions plays an important role in hiring and promotion decisions. In this article we examine the degree of diversity within economics based on the journal quality perceptions of 2,103 AEA economists worldwide. Specifically, we empirically test for factors that might explain differences in an economist&#8217;s journal quality perceptions. These factors include an economist&#8217;s geographic origin, school of thought, journal affiliation, field of specialization and research orientation. Indeed, we find that a significant degree of diversity in journal quality perceptions exists between economists that belong in different subgroups. These results might explain the frequent debates in tenure and promotion committees where journal standings are used for the evaluation of a researcher&#8217;s output.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bo</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1985</link>
		<dc:creator><![CDATA[Bo]]></dc:creator>
		<pubDate>Wed, 30 Aug 2006 12:30:41 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1985</guid>
		<description><![CDATA[In my field we have double blind review (I think and hope) - and I prefer this because, in theory at least, it ensures some kind of fair or due process. If you as a reviewer have time (and find it appropriate) to look through Google for who the author(s) are, then that seems to me to suggest that you are NOT intending to be fair - in essence you are biasing yourself. I have NEVER even considered looking for who the author might be on the web - why would I? Unless I somehow make this a criteria for my decision as to whether or not this paper is quality (sounds stupid to me)..

In fact, I have several times returned the paper to the editor because I suspect I know who the authors are by their sample etc (and I happen to know these people personally). In these cases I do not feel that I should review the paper because I might be biased and would form opinions about my decision based on personal relationships rather than the quality and content of the paper. I know - this is because I am weak and cannot separate my personal life from my professional life etc - but who really can? I think it makes sense to send it back to the editor unless you really think that you are the only one in the world who can review this paper or who is THE expert in the field (sadly I am not - nor do I ever expect to be - one such expert)...]]></description>
		<content:encoded><![CDATA[<p>In my field we have double blind review (I think and hope) &#8211; and I prefer this because, in theory at least, it ensures some kind of fair or due process. If you as a reviewer have time (and find it appropriate) to look through Google for who the author(s) are, then that seems to me to suggest that you are NOT intending to be fair &#8211; in essence you are biasing yourself. I have NEVER even considered looking for who the author might be on the web &#8211; why would I? Unless I somehow make this a criteria for my decision as to whether or not this paper is quality (sounds stupid to me)..</p>
<p>In fact, I have several times returned the paper to the editor because I suspect I know who the authors are by their sample etc (and I happen to know these people personally). In these cases I do not feel that I should review the paper because I might be biased and would form opinions about my decision based on personal relationships rather than the quality and content of the paper. I know &#8211; this is because I am weak and cannot separate my personal life from my professional life etc &#8211; but who really can? I think it makes sense to send it back to the editor unless you really think that you are the only one in the world who can review this paper or who is THE expert in the field (sadly I am not &#8211; nor do I ever expect to be &#8211; one such expert)&#8230;</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bob V</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1959</link>
		<dc:creator><![CDATA[Bob V]]></dc:creator>
		<pubDate>Tue, 29 Aug 2006 17:06:00 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1959</guid>
		<description><![CDATA[&lt;i&gt;my understanding of this process is that part of the reason for having reviewers in the first place is to ensure a fair and essentially BLIND review process.&lt;i&gt;&lt;/i&gt;&lt;/i&gt;

Is this true?  Most of the journals in my field are only single-blind (reviewers know who the author is) rather than double-blind (both reviewers and authors are in the dark).  It would seem to me that the only reason to *not* have a double-blind process is to take the name of the author into account when reviewing the paper.&lt;i&gt;&lt;i&gt; &lt;/i&gt;&lt;/i&gt;]]></description>
		<content:encoded><![CDATA[<p><i>my understanding of this process is that part of the reason for having reviewers in the first place is to ensure a fair and essentially BLIND review process.</i><i></i></p>
<p>Is this true?  Most of the journals in my field are only single-blind (reviewers know who the author is) rather than double-blind (both reviewers and authors are in the dark).  It would seem to me that the only reason to *not* have a double-blind process is to take the name of the author into account when reviewing the paper.<i></i><i> </i></p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Peter Klein</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1956</link>
		<dc:creator><![CDATA[Peter Klein]]></dc:creator>
		<pubDate>Tue, 29 Aug 2006 14:56:33 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1956</guid>
		<description><![CDATA[Bo, two remarks: First, blind reviewing has been rendered technologically obsolete, courtesy of Google. Second, they key point on identities is not that the editor knows the author&#039;s identity, but that the author knows the editor&#039;s identity. Bias or shirking on the editor&#039;s part affects the editor&#039;s reputation. Bias or shirking on the anonymous reviewer&#039;s part has a far smaller effect. (It may affect the reviewer&#039;s reputation with the editor, but not with the author.)]]></description>
		<content:encoded><![CDATA[<p>Bo, two remarks: First, blind reviewing has been rendered technologically obsolete, courtesy of Google. Second, they key point on identities is not that the editor knows the author&#8217;s identity, but that the author knows the editor&#8217;s identity. Bias or shirking on the editor&#8217;s part affects the editor&#8217;s reputation. Bias or shirking on the anonymous reviewer&#8217;s part has a far smaller effect. (It may affect the reviewer&#8217;s reputation with the editor, but not with the author.)</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bo</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1943</link>
		<dc:creator><![CDATA[Bo]]></dc:creator>
		<pubDate>Tue, 29 Aug 2006 09:57:37 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1943</guid>
		<description><![CDATA[Just a quick note to this extremely interesting and relevant discussion.

While I may agree in principle that some editors are delegating too much power and decision-making to their reviewers, my understanding of this process is that part of the reason for having reviewers in the first place is to ensure a fair and essentially BLIND review process. The fact that the editor knows the identity of the author(s) presents somewhat of a problem to me because we all know that (intended or not) human beings tend to be biased - perhaps toward known names? 

If we want truly blind and fair reviews then we need to give most of the power to the reviewers. It is then the job of the editor to ensure that a given paper is reviewed by the best qualified reviewers. In my view - this is where the problem lies: how do we ensure quality reviews - even from the best qualified people (who are busy and may be negatively biased toward anything out of the ordinary: read their own literature tradition)?

In my view, the role of the editor is not only to break up ties but to carefully weigh the comments of the reviewers in order to make the final decision. Reviewers make mistakes. Reviewers may be biased etc. Some arguments may be (and should be) stronger than others. For instance, in many to tier journals you will typically (at least in theory) get three reviewers, where one is a methodologist (may or may not be within your field), one is a theory person (perhaps in your field) and one is a more general type reviewer. How do you distinguish between the comments and recommendations from these three? It may be that the paper is rejected based on a poor methodology, yet the paper makes an extremely interesting theoretical contribution etc. The methods person may not see (or be willing to entertain) the possibility of revising this paper along these lines - even if it does mean a complete methodological makeover - yet if the editor believes in the novelty of the argumentation and that the methodology problems could be solved - perhaps this paper should be given at least a second chance?]]></description>
		<content:encoded><![CDATA[<p>Just a quick note to this extremely interesting and relevant discussion.</p>
<p>While I may agree in principle that some editors are delegating too much power and decision-making to their reviewers, my understanding of this process is that part of the reason for having reviewers in the first place is to ensure a fair and essentially BLIND review process. The fact that the editor knows the identity of the author(s) presents somewhat of a problem to me because we all know that (intended or not) human beings tend to be biased &#8211; perhaps toward known names? </p>
<p>If we want truly blind and fair reviews then we need to give most of the power to the reviewers. It is then the job of the editor to ensure that a given paper is reviewed by the best qualified reviewers. In my view &#8211; this is where the problem lies: how do we ensure quality reviews &#8211; even from the best qualified people (who are busy and may be negatively biased toward anything out of the ordinary: read their own literature tradition)?</p>
<p>In my view, the role of the editor is not only to break up ties but to carefully weigh the comments of the reviewers in order to make the final decision. Reviewers make mistakes. Reviewers may be biased etc. Some arguments may be (and should be) stronger than others. For instance, in many to tier journals you will typically (at least in theory) get three reviewers, where one is a methodologist (may or may not be within your field), one is a theory person (perhaps in your field) and one is a more general type reviewer. How do you distinguish between the comments and recommendations from these three? It may be that the paper is rejected based on a poor methodology, yet the paper makes an extremely interesting theoretical contribution etc. The methods person may not see (or be willing to entertain) the possibility of revising this paper along these lines &#8211; even if it does mean a complete methodological makeover &#8211; yet if the editor believes in the novelty of the argumentation and that the methodology problems could be solved &#8211; perhaps this paper should be given at least a second chance?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: are reviewers too powerful? &#171; orgtheory.net</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1905</link>
		<dc:creator><![CDATA[are reviewers too powerful? &#171; orgtheory.net]]></dc:creator>
		<pubDate>Sun, 27 Aug 2006 23:42:42 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1905</guid>
		<description><![CDATA[[...] Yes, says Nicolai Foss at Organizations &amp; Markets - see his post. [...]]]></description>
		<content:encoded><![CDATA[<p>[...] Yes, says Nicolai Foss at Organizations &amp; Markets &#8211; see his post. [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bob V</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1899</link>
		<dc:creator><![CDATA[Bob V]]></dc:creator>
		<pubDate>Sun, 27 Aug 2006 20:14:53 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1899</guid>
		<description><![CDATA[I have been intrigued by Jung-Chin&#039;s point.  The hands-down most respected journal for folks like me to publish is Management Science.  My perception is that Management Science is a clique.  However, it isn&#039;t as if everyone else gets annoyed and submits their best work to another journal.  The best work still ends up in MS despite its apparent prejudice.  How damaging is this, and how long can it be sustained?]]></description>
		<content:encoded><![CDATA[<p>I have been intrigued by Jung-Chin&#8217;s point.  The hands-down most respected journal for folks like me to publish is Management Science.  My perception is that Management Science is a clique.  However, it isn&#8217;t as if everyone else gets annoyed and submits their best work to another journal.  The best work still ends up in MS despite its apparent prejudice.  How damaging is this, and how long can it be sustained?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Peter Klein</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1895</link>
		<dc:creator><![CDATA[Peter Klein]]></dc:creator>
		<pubDate>Sun, 27 Aug 2006 19:06:52 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1895</guid>
		<description><![CDATA[Further to Jung-Chin&#039;s point, it&#039;s also a matter of Type I versus Type II error. Editors would rather not publish a paper that is later judged to be good than publish a paper that is later judged to be bad.]]></description>
		<content:encoded><![CDATA[<p>Further to Jung-Chin&#8217;s point, it&#8217;s also a matter of Type I versus Type II error. Editors would rather not publish a paper that is later judged to be good than publish a paper that is later judged to be bad.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jung-Chin Shen</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1894</link>
		<dc:creator><![CDATA[Jung-Chin Shen]]></dc:creator>
		<pubDate>Sun, 27 Aug 2006 19:02:43 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1894</guid>
		<description><![CDATA[I guess another possibility is that many editors are risk-averse: They tend not to overrule reviewers in order to attribute the rejection responsibilities to anonymous reviewers. As many journals are actually controlled by a small number of invisible academic community members (To cite an non-management journal, though it is not the real example in my mind because of its single-blind reviewing process, most Quarterly Journal of Economics authors have connections with either Harvard or MIT), such practice may enhance publishing reciprocity behaviors among the members of the community. For example, I know one journal that Nicolai likes very much has ever exhibited such tendency: some of their department editors accept each other’s papers. If quality is solely socially constructed, such practice will not hurt the reputation of the journal and may persist over time.]]></description>
		<content:encoded><![CDATA[<p>I guess another possibility is that many editors are risk-averse: They tend not to overrule reviewers in order to attribute the rejection responsibilities to anonymous reviewers. As many journals are actually controlled by a small number of invisible academic community members (To cite an non-management journal, though it is not the real example in my mind because of its single-blind reviewing process, most Quarterly Journal of Economics authors have connections with either Harvard or MIT), such practice may enhance publishing reciprocity behaviors among the members of the community. For example, I know one journal that Nicolai likes very much has ever exhibited such tendency: some of their department editors accept each other’s papers. If quality is solely socially constructed, such practice will not hurt the reputation of the journal and may persist over time.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Peter Klein</title>
		<link>http://organizationsandmarkets.com/2006/08/27/are-reviewers-too-powerful/#comment-1893</link>
		<dc:creator><![CDATA[Peter Klein]]></dc:creator>
		<pubDate>Sun, 27 Aug 2006 19:02:04 +0000</pubDate>
		<guid isPermaLink="false">https://organizationsandmarkets.wordpress.com/2006/08/27/are-reviewers-too-powerful/#comment-1893</guid>
		<description><![CDATA[Other strong and idiosyncratic editors, besides Clower, include Gordon Tullock at Public Choice and Murray Rothbard at the Journal of Libertarian Studies and the early Review of Austrian Economics. Both used referees sparingly, if at all, confident in their own judgments of quality. Tullock was also legendary for his quick turanounds -- he usually responded to each author, personally, within a day or two of receiving a manuscript.]]></description>
		<content:encoded><![CDATA[<p>Other strong and idiosyncratic editors, besides Clower, include Gordon Tullock at Public Choice and Murray Rothbard at the Journal of Libertarian Studies and the early Review of Austrian Economics. Both used referees sparingly, if at all, confident in their own judgments of quality. Tullock was also legendary for his quick turanounds &#8212; he usually responded to each author, personally, within a day or two of receiving a manuscript.</p>
]]></content:encoded>
	</item>
</channel>
</rss>

