Organizations and Markets

Incoherence Is Bad For You but Good For Us

4 August 2007 at 4:21 pm spostrel 3 comments

| Steven Postrel |

I just finished reading David Hull’s remarkable Science as a Process (1988), and was struck by one of his arguments. One of his claims (not his major thesis) is that while each scientist strives to make his own work coherent and internally consistent, overall progress only occurs because the views of every school of thought and every discipline are somewhat incoherent.

Among the philosophers, sociologists, and historians of science a big argument has swirled over the statement “all observations are theory-laden.” Just about everybody accepts this post-positivist dictum, because it seems so obviously true — for example, what aspects of a phenomenon you pay attention to, out of the zillions of aspects you ignore, depends upon your underlying (if often implicit) theory about the phenomenon and its importance. Should industry evolution be characterized by firm birth and death rates? By growth rates of extant firms? By the spread of routines?

One danger this situation poses is of a self-sealing loop, where theories are accepted because of data whose interpretation depends on the theory itself. In this view, while discrepant observations are thought to be irrelevant or are interpreted away as noise or error, confirming observations are carved out of the noise based on theoretical preconceptions. Classic organization ecology is a theory about the births and deaths of organizations, with “deaths” including selling out at a profit. Trying to refute its ideas about competition and legitimacy by reference to firm profits or growth rates was pointless. A Bayesian might say that strong priors limit the sway of data on our posteriors, but the real issue is deeper than that; our “priors” may influence what we count as a confirming or disconfirming instance in the first place.

Hull argues, however, that variation in beliefs among research schools, as well as rivalry across schools, prevents this self-sealing problem from ever becoming a reality. Even close collaborators disagree about many aspects of their theories (though they may try to publicly suppress these differences for political reasons), so even if one school of thought comes to dominate a discipline, there will always be different “takes” on what counts as data and how data should be interpreted. He empirically demonstrates this claim about disagreement within schools of thought, with detailed evidence from evolutionary biology and systematics. In one striking instance, scientists who were closely identified as theoretical and political allies in the hottest disputes within systematics, joined at the hip so to speak, found themselves unable to co-author a textbook due to internal disagreements.

This incoherence about what theory means and how it should be interpreted ensures that self-sealing beliefs can’t be locked in. It is an irreducible source of evolutionary variation. It’s bad for an individual scientist if his views are incoherent — he’ll get ignored or picked apart — but the overall field avoids getting stuck in self-confirmation by blurriness and disagreement over basic definitions, premises, interpretations, and so on. So while we rightly push for greater coherence in our theoretical structure, our failure to achieve it may make scientific progress more likely in the long run.

Remarkably, Peter Galison, the prominent historian of science, came to a similar conclusion, apparently independently, in his later (1996) book Image and Logic, which studied the development of particle physics from an experimental and instrumental point of view. He argued, against some of his more relativist and pomo colleagues, that the division of particle physics into discrete theoretical, phenomenological, experimental, and instrumental communities prevented the social construction of truth. Experimental physicists, in his account, don’t “construct” the particles predicted by theorists because, to put it bluntly, they don’t even understand most of the things that theorists say (and don’t always care that much).

Famous discoveries, spikes in the data at particular energies and angles, were often complete surprises to the people running the experiments, with explanation and interpretation only coming afterwards when the theorists saw their findings. The “theories” with which these observations are laden are not the high theories that make it onto public television but rather background theories about how apparatus works and noise is generated — an experimenter’s theory distinct from textbook physics. Galison refers to the “intercalated” nature of theoretical, experimental, and instrumental knowledge, like a brick wall where each row of bricks is deliberately not aligned with the row above. Experimental schools of thought are not generally coincident with theoretical ones in physics, so we don’t have to worry too much about self-sealing loops.

As someone often frustrated by the incoherence found in strategy, I find these arguments intellectually though not emotionally comforting. All these aggravating arguments about definitions and interpretations are part of a process that protects us from getting locked into self-sealing loops between theories and data. But why can’t everybody get locked into MY beliefs? That wouldn’t be so bad.

Entry filed under: Former Guest Bloggers, Methods/Methodology/Theory of Science, Myths and Realities, Pomo Periscope.