Posts filed under ‘Methods/Methodology/Theory of Science’
Controversy Over JPE Paper on File Sharing
| Peter Klein |
Stan Liebowitz, no stranger to controversy (1, 2), maintains that the Oberholzer-Gee and Strumpf paper on file sharing, published last year in the Journal of Political Economy, is fundamentally flawed. Stan submitted a comment (longer version here) to the JPE which was rejected by editor Steve Levitt. Stan believes that Oberholzer-Gee and Strumpf are guilty not merely of sloppiness, but academic dishonesty, and is upset that they refuse to share their data. The German newspaper Handelsblatt has written an article about the controversy. Handelsblatt focuses on Levitt’s decision to ask Strumpf to write a reply and then to use the reply as an anonymous referee report in rejecting Liebowitz’s comment. That doesn’t trouble me as much as the authors’ unwillingness to share the data (and the JPE’s refusal to insist on it). More generally, notes the newspaper:
The impression that procedural standards of economics journals are not particularly strict is widely shared in the profession. Zurich-based economist Ernst Fehr, an associate editor of the top-five journal “Quarterly Journal of Economics” and of “Science” points to a lack of clear rules as to when an editor should recuse himself because of potential prejudice. Science journals also seem to deal more openly with the competition among scientists. “Authors who submit an article to a science journal can say who they do not want to review their article”, praises Fehr, a choice which is typically not given to economists.
One internationally renowned economist, who did not want to be named, expresses the complaint more bluntly: “Little scandals and big scandals are commonplace: editors who publish articles in their own journals, referees or editors who decide about articles submitted by their own doctoral students.”
The pointer is from Craig Newmark, who writes: “Until important empirical results in economics are, as a matter of routine, carefully scrutinized and until they are provably replicable, economics will never get the respect that physics and biology and chemistry get. And that’s a shame.”
Update: Additional comments from John Lott and John Palmer.
Academic Journal Fakery
| Peter Klein |
As computer programs make images easier than ever to manipulate, editors at a growing number of scientific publications are turning into image detectives, examining figures to test their authenticity.
And the level of tampering they find is alarming. “The magnitude of the fraud is phenomenal,” says Hany Farid, a computer-science professor at Dartmouth College who has been working with journal editors to help them detect image manipulation. Doctored images are troubling because they can mislead scientists and even derail a search for the causes and cures of disease.
Ten to 20 of the articles accepted by The Journal of Clinical Investigation each year show some evidence of tampering, and about five to 10 of those papers warrant a thorough investigation, says Ms. Neill. (The journal publishes about 300 to 350 articles per year.)
This is from the Chronicle. The problem is partly cultural, it appears. “[Y]oung researchers may not even realize that tampering with their images is inappropriate. After all, people now commonly alter digital snapshots to take red out of eyes, so why not clean up a protein image in Photoshop to make it clearer?” Says Farid: “This is one of the dirty little secrets — that everybody massages the data like this.”
I suspect that outright fraud — making up data, changing regression coefficients — is unusual in empirical social-science research research. Sloppiness, ranging from data-entry errors to programming mistakes to misspecified regression models, is common. And social scientists typically “shade” results, e.g., by running fifty regressions and reporting only the one in which the signs and significance levels turn out to the researcher’s liking. (Hence the growing importance of the “robustness checks” section of any empirical paper.)
Before They Were Famous
| Randy Westgren |
If we point to “The Nature of the Firm” (1937) as the moment when Ronald Coase earned is place in the Pantheon, then we can go back two years (and 16 years before his doctorate was awarded) to a period when he was lecturing at LSE and working on public utilities to find the beginning of a series of papers in Economica (with R.F. Fowler) on the English pig industry (see here, here, here, and here for JSTOR links). The story line is about the effects of anticipated prices (based on lags) for hogs and corn on production decisions and the consequent cobweb model of dynamic prices. A classic in agricultural economics.
Another giant figure, Sewell Wright, examined the same phenomenon in the US in an obscure publication ten years earlier: Corn and Hog Correlations, USDA Department Bulletin # 1300, July 1925. Wright was an animal husbandman (and guinea pig breeder) at USDA after completing a doctorate in genetics at Harvard. (more…)
Interdisciplinary Degree Programs
| Peter Klein |
ANAKIN: How do you know the ways of the Force?
PALPATINE: My mentor taught me everything about the Force . . . even the nature of the dark side.
One of the great things about being Oliver Williamson’s PhD student is that he encouraged us to read widely outside our core discipline. His “Economics of Institutions” course included materials from political science, business history, cognitive science, and even . . . sociology! So while my primary training was in economics, I was exposed to some of the best thinkers in the “contiguous disciplines.” I even know about the dark side.
Nonetheless, I was enrolled in a traditional degree program and was credentialed in an established discipline (economics). Recently I’ve been talking to colleagues in several departments about the possibility of creating a truly interdisciplinary social-science degree program, one that would train students in two or more disciplines organized around core subject areas like organizational studies, entrepreneurship, regional development, and the like. This sort of curriculum has obvious potential advantages in fostering intellectual breadth, developing critical thinking skills, and encouraging relevance and accessibility beyond a handful of specialists. On the other hand, there are many potential hazards. The training may be broad but not deep; opportunities for narrow, technical advances may be missed; students may struggle to find academic placements; and so on.
I’d like to hear from readers who have experience with, or informed opinions on, such interdisciplinary programs. The best-known is probably Chicago’s Committee on Social Thought, which included T. S. Eliot, F. A. Hayek, Allan Bloom, Saul Bellow, Hannah Arendt, and Leo Strauss on its faculty and produced outstanding graduates such as Ralph Raico and Ronald Hamowy. Arizona State’s School of Global Studies is a more recent example. Vanderbilt recently started a PhD program in law and economics. What are some other programs I should be studying?
Peter and Inspiration
| Randy Westgren |
Before enplaning for Vancouver, I spent a great day at the University of Missouri with Peter Klein and his (local) colleagues. I discovered that Peter and I share a common interest in the fiction of Richard Powers, a novelist whose works draw from the biological, physical, cognitive, and information sciences. Moreover, Peter acknowledged that his favorite Powers novel is The Gold Bug Variations; it’s my favorite as well. I finished my second reading on the airplane and found a passage that incites this post:
The world we know, the living, interlocked world, is a lot more complex than any market. The market is a poor simulation of of the ecosystem; market models will never more than parody the increasingly complex web of interdependent nature. (First edition, p. 411)
I agree that market models are pale abstractions compared to any ecosystem. But I have studied a great many models of ecosystems (dynamic system simulations, agent-based simulations, statistical models of species interactions, analytical models of populations) and find them to be pale abstractions of ecosystems, as well. I will propose — for refutation — that most market models I see are less interesting than ecosystem models; they are still undersocialized in the Granovetter sense. The ecological models seem to require more attention paid to the social interactions of the individuals.
Just a thought.
NIE Workshop for Law Professors
| Peter Klein |
The University of Colorado invites law professors to a one-day workshop, 11 June 2008, on the new institutional economics. Speakers are Lee Alston, Lynne Kiesling, Gary Libecap, Henry Smith, and Tom Ulen. Contents include:
(1) an introduction to NIE and why it matters to legal scholarship, particularly for property and intellectual property law; (2) an introduction to behavioral economics and experimental economics, including a simulation exercise that will demonstrate how experimental economics can be used to examine institutions in practice; and (3) an interactive discussion where all participants examine some case studies to evaluate the payoffs of using NIE and experimental economics to evaluate the merits of different legal regimes.
Sounds like fun (but where’s the theory of the firm?). Thanks to Thom Lambert, one of the lucky attendees, for the heads-up.
Occupational Psychosis
| Randy Westgren |
One of the profoundly valuable benefits of recently giving up an administrator’s position is that I have time to read. I sat down with a stack of journals, biographies, fiction, and cookbooks that has grown since last summer. In the first pass through the stack, I found a couple pieces that echo one of the themes of this blog: how our training affects our perceptions of theory, facts, and phenomena.
One piece is an article by two young, interesting colleagues, Brianna and Arran Caza, who write about “Positive Organizational Scholarship” (POS) in the March 2008 issue of the Journal of Management Inquiry . They argue that the bulk of research on organizations, as highlighted by the top-cited articles in three years of ASQ and AMJ, begin with negative framing of organizational issues — what Brianna and Arran call a deficit model approach. They propose the need for research based on positive framing — not exclusively — as necessary to advance theory and practice in the organizational sciences. The POS paradigm is unabashedly post-modern (up periscope!), but it serves us all when alternative lenses are trained on issues that we all observe from our particular perspectives. (more…)
Best Anti-IRB Article You’ll Read Today
| Peter Klein |
It’s Zachary Schrag’s “How Talking Became Human Subjects Research: The Federal Regulation of the Social Sciences, 1965-1991,” forthcoming in the Journal of Policy History.
In universities across the United States, institutional review boards, or IRBs, claim that they have the moral and legal authority to control the work of researchers in the humanities and social sciences. While IRBs may claim powers independent of federal regulations, they invariably point to these regulations as a key source of their authority. This article draws on previously untapped manuscript materials in the National Archives to trace the history of the federal regulation of social science research. Officials raised sincere concerns about dangers to participants in social science research, especially the unwarranted invasion of privacy as a result of poorly planned survey and observational research. On the other hand, the application of the regulations to the social sciences was far less careful than was the development of guidelines for biomedical research. Regulators failed to define the problem they were trying to solve, then insisted on a protective measure borrowed from biomedical research without investigating alternatives.
See also Schrag’s valuable Instituitional Reveiw Blog.
IRB oversight is particularly strong at the University of Missouri, across all departments, partly the result of a federal investigation in 1999 that came down hard on the medical school. One might wonder what this has to do with social-science research, but there you go.
Authors@Google
| Peter Klein |
From Marshall Jevons I just learned about the Authors@Google lecture series. Lots of good stuff there. The O&M crowd may especially enjoy the talks by Ian Ayres, Larry Lessig, Bob Litan, Richard Florida, John Searle, Daniel Solove, Steven Pinker, Robert Frank, Don Tapscott, Bill Easterly, and Tom Perkins.
Update: If you like this sort of thing check out TED as well (thanks to Art Carden for the pointer). The first person I saw when I visted the site yesterday was Yochai Benkler, whose book The Wealth of Networks I happen to be reviewing for The Independent Review.
Erasmus Journal for Philosophy and Economics
| Peter Klein |
The Erasmus Journal for Philosophy and Economics is a new journal focusing on economic methodology, the history of economic thought, relationships between economics and other disciplines, and similar “meta” issues. Here’s the call for papers for the inaugural issue. The journal advertises that it is “particularly friendly to Young Scholars (graduate students and recent PhD graduates).”
Do Economists Believe in “Atomistic Individualism”?
| Peter Klein |
To many critics economics goes astray in characterizing people as isolated, autistic, self-interested, individualistic utility maximizers, unconnected from the broader social fabric in which they are embedded. The celebrated Ferraro, Pfeffer, and Sutton paper (AMR, 2005), and the broader “performativity” critique of economics, is a typical example of this attitude. Some heterodox economists even long for a “post autistic” version of the discipline.
As emphasized repeatedly on this blog, however, the criticism is fundamentally mistaken. At heart, it confuses methodological individualism with ontological individualism. The assumption of individual utility maximization, the simplified model of an isolated individual, and the like are principles of explanation, not descriptions (or, a fortiori, prescriptions). Now, I do think that economists have gone astray by emphasizing “rationality,” modeled with consistent preferences, a utility function that is monotonic and non-decreasing, etc., rather than the broader concept of “purposeful action,” as Mises described it, which is what most economists before the formalist revolution seemed to have had in mind. (more…)
No Country for Old Probability Theorists
| Peter Klein |
I finally got around to seeing No Country for Old Men, which I enjoyed despite unrealistically high expectations (movies too suffer from the winner’s curse). Javier Bardem’s Anton Chigurh surely belongs with Darth Vader, Hannibal Lecter, Dr. Christian Szell, Nurse Ratched, and Max Cady on the list of all-time great movie villains. The movie is in one sense a meditation on the role of chance in human affairs, so naturally I started thinking about risk, uncertainty, choice, delegation, and other issues near and dear to our organizational hearts.
Chigurh, the cold-blooded killer, likes to flip a coin before deciding whether to kill someone, forcing the victim to call the toss. This reminded me that risk and Knightian uncertainty aren’t mutually exclusive determinants of economic outcomes. Entrepreneurs choose to invest in risky projects, but project selection itself reflects the bearing of Knightian uncertainty. Richard von Mises gives the example of champagne bottles that burst while in storage with predictable frequencies. The champagne producer can quantify the risks associated with bottling and storage. But the choice of producing one variety or another, hiring one type of laborer or another, and even being in the champagne business at all, involves another kind of uncertainty, one that cannot be described with mathematical precision. The decision to enter the champagne business involves Knightian uncertainty, but once that decision has been made, some of the variation in outcome can be characterized as probabilistic risk. Think of it in terms of mixed strategies; the specific move is random, but the decision to play a mixed strategy is not. Likewise, Chigurh can hardly claim that his victims’ deaths are random. A coin flip determines their fate, but he chooses to flip the coin — and that choice cannot be explained by a known probability distribution. (more…)
Numbers Don’t Lie — Or Do They?
| Peter Klein |
Quantitative analysis leads to superior decision making, says Ian Ayres in Supercrunchers. Enthusiasts for expert systems are skeptical of “intuitive” reasoning. And most contemporary social scientists can’t conceive of a world without econometrics, sociometrics, psychometrics, and fill-in-the-blank-ometrics. Even management scholars are getting into the act. Of course, quantitative analysis is only as good as the assumptions that go into it. And economists such as Knight and Mises maintain that some kinds of human decision-making defy quantification and systematization and are fundamentally qualitative, or verstehende (explaining why some entrepreneurs earn profits while others make losses).
Wharton’s Gavin Cassar studies nascent entrepreneurs (defined here as firm founders) and finds, surprisingly, that those who use common accounting practices such as budgeting, sales forecasting, and financial planning are more likely to overestimate future performance than those who rely on qualitative, intuitive projections. “[T]hose individuals who adopt an inside view to forecasting, through the use of plans and financial projections, will exhibit greater ex-ante bias in their expectations. Consistent with inside view adoption causing over-optimism in expectations, I find that the preparation of projected financial statements results in more overly-optimistic venture sale forecasts.” In other words, quantitative analysis may exacerbate, rather than mitigate, cognitive bias. Worth a read (and see this summary in Knowledge@Wharton).
A New Explanation for Scholarly Productivity
| Peter Klein |
I always suspected it: scholarly productivity is inversely related to — beer. That’s the finding of a new study of Czech ornithologists, as summarized in yesterday’s N.Y. Times (thanks to Brian McCann for the heads-up). The more beer a scientist drinks, the less likely he is to publish or to have his work cited. Apparently this is a cross-sectional result, without fixed effects or instrumental variables, so there is little information on causality. Perhaps unsuccessful Czech scientists tend to drown their sorrows at the local pub (no doubt drinking their copycat Budvar). Personally, I am more likely to grab a brew to celebrate the occasional citation, so I’d expect the correlation (under reverse causality) to run the other way. And what about these rats?
The Rhetoric of Science
| Peter Klein |
Tom Lessl, who’s work on the history of science we’ve mentioned before, offers some interesting reflections on scientific rhetoric in this 2005 interview.
There is a popular and widespread misconception in the world that scientific communication is distinctly different from other forms of public communication, but this is not really so. Its persistence is explained by an old adage in my field, which I think comes from Roderick Hart at the University of Texas, which says that rhetoric is most effective which disguises itself as something else. And I would have to say that science is the master of disguises. . . .
In saying this I am not trying to suggest that science is not a profoundly powerful form of inquiry, that its truth claims are without substance or that many scientific questions cannot be answered with a definitive yes or no. But scientific communication has all the same kind of properties that we typically find in other arenas of communication.
This misconception, Tom argues, is actively promoted by scientists themselves, primarily as a means of securing resources:
What I call science’s “priestly voice” is the outcome of several hundred years of experimentation with different ways of relating itself to its patrons. Patronage is a perennial problem for science, one of huge proportions. Science is at once an exceedingly costly undertaking and also one that does not necessarily offer any immediate return on investments. We all know that science has produced applications of immeasurable benefit, but in history when scientific patronage has been dependent upon the promise of such payoffs, science work has suffered. This is because most of what we call basic science is exploratory and can’t promise applications. It produces knowledge that winds up in science journals but not in pharmaceutical patents or medical applications. The characteristic expectation of Americans that science is valuable because it pays off has traditionally deterred scientific growth. This was why the U.S. remained a backwater province of theoretical science until after WWII — when the public began to realize that theory might pay off in things like atom bombs. But more generally, scientific culture has responded to the pressures of patronage by trying to construct a priestly ethos — by suggesting that it is the singular mediator of knowledge, or at least of whatever knowledge has real value, and should therefore enjoy a commensurate authority. If it could get the public to believe this, its power would vastly increase. (more…)
Nietzsche and Contemporary Philosophy
| Peter Klein |
“Nietzsche is peachy,” according to a bumper-sticker I once saw. Nietzsche is sometimes cited in management research as an authority on power, complexity, time, or relativism (e.g., Singer, 1994; Kilduff and Mehra, 1997; Mainemelis, 2001). But what did Nietzsche really say about these things? What are his main contributions to philosophy? Professional philosophers can’t seem to agree, as witnessed in this roundtable conversation with Peter Bergmann, Teodor Münz, Frantisek Novosád, Paul Patton, Richard Rorty, Jan Sokol, and Leslie Paul Thiele. Bergmann calls Nietzsche “the culture hero of modernism, a cultural revolution comparable to the Reformation or the Enlightenment. His critique of herd values is reflected in the posture of the avant-garde: elitist to the present, democratic to the future.” But Nietzsche was no nihilist, says Sokol; he was rather “an excessively sensitive person horrified by a world where nothing has rules and stands for nothing.”
All agree that Nietzsche bears no personal responsibility by the appropriation of his ideas by German nationalists, but Schrift notes that Nietzsche “chose to write in a style that invites misunderstanding — his use of metaphor, dissimulation, and hyperbole in particular, all make it easier for his words to be taken to mean something other than what he might have intended.” A warning to those of us who like jargon and are guilty of bad academic writing. (HT: 3quarks)
The Role of Economic Analysis in Public Policy
| Peter Klein |
Here are two views on the role of economic analysis in public policy, from a passage in Robert Dodge’s biography of Thomas Schelling recounting the early days of Harvard’s Kennedy School of Government:
A tension [among the faculty] began to develop over an ideological difference between two groups. The question that brought about the division concerned the proper role of the policy analyst. Schelling’s view was the same as it had been since the Cold War, and there were other economists in the school who generally agreed. They believed that the approach to policy analysis was to begin by rationally analyzing situations, seeking to understand how things work and what outcomes would be. His idea had been to “solve the puzzle first.” Policy was something that came after understanding. Throughout his career Schelling had fought against the idea of beginning with outcomes, what he saw as looking at problems backward, and had believed that strategic analysis was required in advance to understand situations before developing public policy.
A group headed by Steve Kelman and future Secretary of Labor, Robert Reich, held a different view. This group cared about policy management as well as public policy analysis. Their main argument was that values couldn’t be separated from public policy, and the antiseptic and purely analytical approach of Schelling’s group was incomplete. Policy analysis, the Reich group felt, was to be used in determining a successful path to the goal one hoped to achieve. They believed it was necessary to acknowledge and identify openly what one was trying to achieve or affirm when carrying out a policy. . . . (more…)
Special Issue of JEM on Thomas Schelling
| Nicolai Foss |
Nobel Prize winner (2005) Thomas Schelling makes social science come alive. He has contributed fundamental insights to game theory (e.g., the notion of a focal point, the importance of commitment, early insights in the epistemic conditions of Nash equilibrium, signaling, etc.) and to the understanding of social dynamics (e.g., the famous 1971 checkerboard segregation model; early insights in “critical mass” and “tipping”). He is among the founders of game-theoretic conflict theory.
Schelling has an amazing knack for drawing fundamental lessons from simple illustrations. He rarely uses advanced mathematics, he is more interested in processes than in equilibrium states, and substantial parts of his work is accessible to the educated layman (e.g., this one and this one). He is quite an unusual social scientist.
The latest (Dec.) issue of the ever-interesting Journal of Economic Methodology features a Symposium on Thomas Schelling edited by Abu Rizvi (who, in other journals and volumes, has published some of the most penetrating meta-theoretical work on game theory). (more…)
Workshop on Performativity and Finance
| Nicolai Foss |
We have blogged occasionally on performativity and related issues here at O&M (e.g., here), though by no means with the frequency of the chaps at our evil twin, orgtheory.net (here is a sample of their 1,206 posts on the subject).
Daniel Beunza (Columbia) and Yuval Millo (LSE) are organizing a workshop 28-29 April on performativity and finance, one of the main areas of applications of the performativity concept. The title of the workshop? Well, “From Bodies to Black-Scholes” (what else?). (more…)
DeLong on Introspection
| Peter Klein |
Brad DeLong sounds almost Misesian in this call for economics PhD students to study economic history:
[Mainstream] Economics is the hyper-positivist of social science disciplines: believing that everything of interest can be reduced to law-like theoretical and empirical propositions modeled after classical mechanics; that what cannot be reliably, repeatedly, quantitatively, and empirically demonstrated does not really exist as knowledge; that the only good social science is a deductive, analytical, model-based, general, experimental science.
But this misses a lot. Because we are people like those whom we study, we have psychological access to our subjects’ internal decision-making processes and motivations at a level that we cannot obtain from market price-quantity data. There is lots of interest that happens once and only once. Natural experiments are rare, and so if we restrict ourselves to positivist tools alone much is underidentified. The individuals’ preferences — the “tastes” part of “tastes and technologies” are not primitive but are themselves the result of long and complex historical, sociological, psychological, and — yes — economic processes. You need thickly-described case studies and anecdotes looking out from people’s insides before you can tell if your statistical results mean what you assert they mean.
Mises argues in Theory and History (1957) that the basic economic categories of means and ends, of preference, contraint, and choice, cannot be understood in purely positivist terms: “Being himself a valuing and acting ego, every man knows the meaning of valuing and acting. He is aware that he is not neutral with regard to the various states of his environment, that he prefers certain states to others, and that he consciously tries, provided the conditions for such interference on his part are given, to substitute a state that he likes better for one he likes less.” In other words, we understand economic activity in a causal, realistic sense, a sense denied to us in our study of the natural world. Moreover, like Brad, Mises argued that the historian must use not only the tools of deductive theory, but a deep understanding or Verstehen, to grasp the meaning of particular historical events. (more…)
Organizations and Markets 
Recent Comments