Weakly Informative Priors
11 May 2007 at 8:51 am Peter G. Klein 5 comments
| Peter Klein |
I clearly remember an incident from my first week of graduate school. I asked Professor X if he thought I should take Professor Y’s econometrics course. “Well, Y is a good teacher,” X replied. “Of course,” he added quietly, with a conspiratorial glance, “you know he’s a Bayesian, right?”
“A Bayesian? My goodness, I had no idea!” I exclaimed, not having the faintest idea what a Bayesian was. Professor X said it the way he might have said “wife-beater,” so I was sure I wanted nothing to do with such a character. Later, after studying Bayesian inference, Bayes’s Theorem, the Bayesian approach to games of incomplete information, and the like, I came to regard Bayesians as a bit less dangerous, something like the eccentric uncle at a family reunion that everyone tolerates but tries to avoid.
One controversial issue in the Bayesian approach to games of incomplete information is the assumption that agents share common priors. A related issue is that priors, whether modeled as independent or shared, are assumed to be either completely true, or completely false. Columbia’s Andrew Gelman and Aleks Jakulin present al alternative approach, which they term “weakly informative priors.”
Bayesians traditionally consider prior distributions that (a) represent the actual state of subject-matter knowledge, or (b) are completely or essentially noninformative. We consider an alternative strategy: choosing priors that convey some generally useful information but clearly less than we actually have for the particular problem under study. We give some examples, including the Cauchy (0, 2.5) prior distribution for logistic regression coefficients, and then briefly discuss the major unsolved problem in Bayesian inference: the construction of models that are structured enough to learn from data but weak enough to learn from data.
See also Robin Hansen’s comments on common priors.
Entry filed under: - Klein -, Methods/Methodology/Theory of Science.
Organizations and Markets 
1.
Joseph Mahoney | 11 May 2007 at 10:46 am
I take it then that Professor Y had a low Bayesian prior concerning the soundness of Bayesians.
I wonder how Professor Y would respond to the assertion that at best, on balance, our empirical studies change our Bayesian priors for the better.
The second edtion of Richard Lipsey’s AN INTRODUCTION TO POSITIVE ECONOMICS said it better:
“I have abandoned the Popperian notion of refutation and have gone over to the statistical view of testing that accepts that neither refutation nor confirmation can ever be final, and that all we can hope to do is to discover on the basis of finite amounts of imperfect knowledge what is the balance of probabilities between competing hypotheses” (Lipsey, 1966: p.xx).
2.
Tom S. | 11 May 2007 at 6:53 pm
That Lipsey quote is interesting*, but I ultimately believe is wrong. We know a lot of things which aren’t (e.g., geocentricism). Further, it’s hard to see where his epistemology makes room for the notions of hypotheses.
*Interesting enough where I’m now looking a buy a copy.
3.
Rafe | 15 October 2009 at 5:33 pm
Pity Lipsey never understood Popper on testing.
4.
bill wald | 5 January 2010 at 12:46 pm
Herman Dooyeweerd (see Google) makes a good case that all humans are hard wired by God for religion and make something into a religion for themselves. He says it nicer.
On the other hand, “If atheism is a religion then bald is a hair color.”
5. On the Term “Religion” | The Beacon | 7 January 2010 at 12:03 pm
[…] had similar experiences. Thank goodness for that twelve-step […]