The notion of using inaccuracies in self-reports of biographical or demographic information to proxy for a knowledge-based variable is something I’ve tried in my research as well. Recently, I have used a particular dataset where tens of thousands of individuals trained in science or engineering have been surveyed about their work history and educational backgrounds. Believe it or not, there is a small percentage of individuals who actually report a set of majors and degrees in one year, and then two years later omit (or report as a replacement) a different major or degree. My argument was that this indicated deficiencies in respondents’ long-term memory retrieval capabilities, since the list of categories of majors never changed from one survey administration to the next.

I found evidence suggesting that my proxy for long-term memory capabilities was reasonably accurate, but more than a few dear colleagues have found it difficult to accept. When I tried to explain to them that some individuals trained in science or engineering appeared to have trouble recalling their major over the years, they looked at me quizically (”That’s impossible!”)… even when I showed that the coding scheme used to enter the survey data left data entry error highly unlikely.

I would guess that these same good colleagues of mine would have a difficult time accepting that people can’t reliably report their own age!