Evaluating New Ideas: Looking Across and Looking Beyond
| Nicolai Foss |
Those of us who have experience with research councils and other funding bodies with expert evaluations of the submitted research are familiar wilth folklore, such as “When evaluating economists routinely smash non-economics projects,” “sociologists are a total incrowd and will not tolerate any application of rational choice method, serious econometrics or common sense,” etc. Of course, this is part of the various conspiracy theories about how, notably, economists seek to establish intellectual hegemony.
However, the folklore may be wrong. In a new paper, “Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science,” Kevin Boudreau, Eva Guinan, Karim Lakhani and Christoph Riedl look at the grant proposal process at a major research university and show that evaluators tend to treat proposals more harshly the closer they were to their own areas of expertise. However, evaluators also treat highly novel proposals negatively. Taking issues of ecological and external validity into account, there are obvious implications for the understanding of the nature of the exploitation/exploration tradeoff: There may indeed be a bias against exploring in the domains of highly novel ideas (as predicted by the literature), but the harsh evaluation of new, but well understood ideas may mean that there is a domain of relatively novel and less well understood ideas within which firms will explore.