Against Freakonomics

10 September 2006 at 8:25 am Peter G. Klein Leave a comment

| Peter Klein |

We’ve discussed the Freakonomics phenomenon on these pages, but said little about the substance of Levitt and Dubner’s famous book. (Nicolai offered some brief remarks here — scroll down to the 15 August entry — in the dark days before Organizations and Markets.)

Several new critiques are floating around. Gene Epstein’s book Econospinning: How to Read Between the Lines When the Media Manipulate the Numbers challenges the chapters on real-estate agent services and the abortion-crime link. And from Division of Labour I learn that Ariel Rubinstein reviewed the book, quite harshly, for Haaretz. Rubenstein thinks the book is clever and entertaining, but not really economics. He also takes Levitt and Dubner to task for playing fast and loose with the facts. Excerpt from Rubenstein’s review:

“Teachers and criminals and real estate agents may lie, and politicians, and even CIA analysts. But numbers don’t.” (17) The reader wonders: “How can … data be made to tell a reliable story?” (161) And Levitt responds: “By subjecting it to the economist’s favorite trick: regression analysis. No, regression analysis is not some forgotten form of psychiatric treatment. It is a powerful –- if limited –- tool that uses statistical techniques to identify otherwise elusive correlations.” (161) This is a curious statement in light of the fact that Levitt is aware of the problematic nature of statistical analysis, acknowledging: “I just don’t know very much about the field of econometrics” (x) and in general thinks that “regression analysis is more art than science.” (163). This is perhaps the central contradiction in the book: On one hand, a recognition of the limitations of statistics, and on the other hand, using it as a magician’s box.

And this:

Take, for example, Levitt’s tales of the big city. The Chicago Municipality administers an annual test for schoolchildren. A suspicion arose that teachers, were “correcting” their students’ answers before sending the tests to be checked. Levitt obtained the data from the municipality and developed a computer program that looks for classes with suspicious combinations of answers. For example, if all of the students in a particular class responded correctly to questions 7, 8 and 10, and erred on question 9, a suspicion arises that the teacher falsified the answers to four questions. (On question 9, the teacher either made a mistake himself or tried unsuccessfully to avoid raising suspicion.) In this way, Levitt discovered dozens of deceitful teachers. The IDF’s intelligence units and credit card companies use similar algorithms. What have we learned about Levitt? He is a smart guy with connections in the municipality. What is the connection to economics? None.