“Illusions in Regression Analysis”
26 December 2011 at 12:02 pm Peter G. Klein 4 comments
| Peter Klein |
Apropos Lasse’s post, check out Scott Armstrong’s “Illusions in Regression Analysis,” via Craig Newmark, who highlights passages like this:
This illusion [that correlation implies causality] has led people to make poor decisions about such things as what to eat (e.g., coffee, once bad,is now good for health), what medical procedures to use (e.g., the frequently recommended PSA test for prostate cancer has now been shown to be harmful), and what economic policies the government should adopt in recessions (e.g., trusting the government to be more efficient than the market).
And this:
Do not use regression to search for causal relationships. And do not try to predict by using variables that were not specified in the a priori analysis. Thus, avoid data mining, stepwise regression, and related methods.
Entry filed under: - Klein -, Management Theory, Methods/Methodology/Theory of Science, Myths and Realities.
Organizations and Markets 
1.
Julian Silk | 27 December 2011 at 9:39 am
Regressions provide suggestions, not proofs, unless two sides agree that a particular regression format is unquestionably valid. The question of government vs. the market depends on the particular question asked and what the circumstances are. In “Two Lucky People”, Milton Friedman and Rose Friedman themselves wrote about being influenced by the New Deal atmosphere of the 1930s. It’s often argued that different monetary policy and changes in the price policy of the NRA would have made an enormous difference, but France, which behaved as the U.S. supposedly should have, had a horrible experience in the Depression. In the absence of cross-section evidence, you want to get an idea of what quantitative effects changes will make, and that’s what regression does for you.
2.
Rafe | 28 December 2011 at 4:45 am
A couple of supports. Back in the 1970s when a manager went overseas and dumped a project in my lap – a regression model of NSW hospital costs – one of the references warned of the dangers of high correlation among the dependent variables,or high correlation between one of the dependdent variables and the dependent. This can produce really weird coefficients, for example a factor which clearly has a positive impact can have its coefficent turn negative when you insert something else that has a really high correlation with the dependent variable.
The other support is a paper by Dani Rodrik.
Click to access rodrik.pdf
“Governments use policy to achieve certain outcomes. Sometimes the desired ends are worthwile, and sometimes they are pernicious. Cross-country regressions have been the tool of choice in assessing the effectiveness of policies and the empirical relevance of these two diamatrically opposite views of government behavior. When government policy responds systematically to economic or political objectives, the standard growth regression in which economic growth (or any other performance indicator) is regressed on policy tells us nothing about the effectiveness of policy and whether government motives are good or bad.”
The methodological problems are aggravated in studies of developing countries because the data are often bad verging on useless.
3. Browsing Catharsis – 12.28.11 « Increasing Marginal Utility | 28 December 2011 at 12:05 pm
[…] A number of criticism of regression I haven’t heard before. I don’t know what I think about them yet. At one point, the author tells us that we should never include more than three variables in a regression when forecasting. I wonder if such a suggestion is worthwhile for forecasting, but limiting the number of control raise my concerns about endogeneity. The article also really stinks of methodological instrumentalism. […]
4. Simoleon Sense » Blog Archive » Weekly Roundup 159: A Curated Linkfest For The Smartest People On The Web! | 1 January 2012 at 4:33 pm
[…] “Illusions in Regression Analysis” – via organizationsandmarkets.com – This illusion [that correlation implies causality] has led people to make poor decisions about such things as what to eat (e.g., coffee, once bad,is now good for health), what medical procedures to use (e.g., the frequently recommended PSA test for prostate cancer has now been shown to be harmful), and what economic policies the government should adopt in recessions (e.g., trusting the government to be more efficient than the market). […]