February 2006
Volume 19, Number 2
Correlation Still Isn't Causation
In his letter to the editor [Observer, December 2005], Justin M. Joffe took issue with my assertion in a “Teaching Tips” column [Observer, September 2005] that “distinguishing correlation from causation” is a crucial critical-thinking skill that all psychology instructors should impart. Joffe went so far as to label my claim “pseudoscientific” and “at best a half truth” and therefore “a half lie.” Lest puzzled readers of the Observer conclude that psychology teachers no longer need be concerned about teaching their students the difference between correlation and causation, we should examine Joffe's arguments carefully.
In fact, Joffe's claims are undermined by two serious errors in reasoning. First, Joffe asserts that it is often possible to rule out with considerable certainty the presence of plausible third variables in correlational designs, and that in such cases one can be “about as sure of causal relationships as you are with most experiments.” Here, Joffe commits the mistake, increasingly common in postmodernist circles, of neglecting to distinguish among research designs that occupy different locations in the hierarchy of evidentiary certainty. Joffe is correct that one can often generate plausible causal hypotheses from correlational designs, but he is mistaken that these designs frequently permit causal inferences as definitive as those of experimental designs. The central problem with conclusively inferring causality from correlational designs is that such designs are bedeviled by a plethora of potential third variables, which the investigator may not have measured or even thought to measure (Meehl, 1970). Joffe's argument presumes omniscience on the part of the researcher — namely, that he or she is aware of all the relevant third variables that could complicate the interpretation of a correlational study's conclusions; a presumption that is rarely, if ever, warranted.
Indeed, the very example Joffe proffered as an illustration — a positive correlation between babies' birth weights (Variable B) and their maternal grandfathers' incomes (Variable A) — reveals the flaw in his argument. Joffe appears to maintain that because Variable A precedes Variable B, we can conclude with reasonable certainty that A caused B, albeit indirectly. By drawing this conclusion, Joffe commits the post hoc, ergo propter hoc (after this, therefore because of this) error. The fact that A precedes B does not necessarily imply a causal relation between them. For example, the fact that the recent spate of television crime shows preceded the recent increase in Atlantic hurricanes does not imply that television crime shows produce hurricanes. Moreover, Joffe's assertion that the correlation between babies' birth weights and their maternal grandfathers' incomes allows us “to rule out, with some plausibility, a third variable” ignores the myriad of viable-third variable explanations that could account for this finding. To take just one example, long-term familial residence in a grossly impoverished neighborhood marked by poor prenatal nutrition, could readily give rise to both low maternal grandfather income and low grandchild birth weight. Moreover, Joffe is incorrect that this correlation is incompatible with indirect genetic influences on both variables.
Second, Joffe argues that distinguishing correlation from causation presumes a simplistic model of causation, specifically, that the causal influence must be both necessary and sufficient. He is mistaken. As Meehl (1977) observed — in an article that should be required reading for all psychologists — causal influences span the gamut from strong (e.g., an etiological factor that is both necessary and sufficient) to moderate (e.g., an etiological factor that is necessary but not sufficient) to weak (e.g., an etiological factor that is neither necessary nor sufficient but that increases overall risk for a condition). There is nothing inherent in the term “causation” that forces us to adopt a simplistic model of etiology, nor does the didactic task of distinguishing correlation from causation preclude us from discussing more nuanced models of causality.
Joffe and I are in complete agreement that psychology instructors must do more than chant the mantra of “correlation isn't causation,” and that they should encourage students to recognize that causation is a more complex and multifaceted concept than meets the eye. Nevertheless, I am happy to reassure concerned readers of the Observer who were about to discard their old statistics lecture notes that they can safely continue teaching their students to distinguish correlation from causation.
— Scott O. Lilienfeld
Emory University
References
- Meehl, P.E. (1970). Nuisance variables and the ex post facto design. In M. Radner & S. Winokur (Eds.), Minnesota studies in the philosophy of science: Vol. IV. Analyses of theories and methods of physics and psychology (pp. 373-402). Minneapolis: University of Minnesota Press.
- Meehl, P.E. (1977). Specific etiology and other forms of strong influence: Some quantitative meanings. Journal of Medicine and Philosophy, 2, 33-53.
Consequences of a Self-Appraisal,
Self-Correcting Approach to Exams
I read with interest Joann M. Montepare's article “A Self- Correcting Approach to Multiple Choice Tests” [Observer, October 2005]. Montepare's suggestion, to return students their exams, so that they may reflect on errors and search for the correct answers at home, seems well justified, bold, and efficient. I agree with Montepare that an exam represents not only an opportunity to test what a student knows but also a resourceful learning opportunity for both teachers and students.
However, in my experience — teaching at the college level since 1992 — I have used self-correction only in class, never as an out-of-class task. As such, there are a few notable differences between my procedure and Montepare's. The first and most important difference refers to the use of a self-appraisal, which is carried out before the self-correction. Specifically, after each exam but before one knows the actual results, each student is asked to make a self-appraisal of his or her performance. The student must guess the ratio between his or her right and wrong answers for both multiple-choice and essay questions. The rationale of this pre-correction self-appraisal is to determine how accurate the student's image about the test results is immediately after he or she has taken the exam. The student's selfcorrection is always followed by a teacher's correction and feedback in the next class meeting.
When compared with the real results of the exam, self-appraisals may fall in one of the following categories: first, under-evaluation, when the perception of one's performance is significantly lower than the actual performance; second, over-evaluation, when the perception of one's performance is significantly higher than the actual performance; third, a fair, objective self-evaluation, when one's perception is very close to the actual performance.
A student's auto-classification in one of these categories becomes evident after the self-correction and he or she has received feedback. Each student then has an opportunity both to know his or her actual results and to measure the accuracy of the performance appraisal.
How important are these two elements for the meta-cognitive processes of learning? The procedure of self-evaluation — validated through self-correction and based on real outcomes — is conceptually rooted in the classical distinction made by James (1890/1983, pp. 279- 285, p. 350) between self as knower (“I”) and self as known (“Me”) and Jung's idea on ego-inflation and its relation to the inner voice (1934/1977, pp. 184-185; 1943/1977, p. 71).
Practically, the teacher must explain this procedure on the first day of class and before each exam. Also, students should be told that as the individual's knowledge in a given field increases, the perception of one's anticipated and actual performance should become closer to the actual result.
If self-evaluations are a bit higher, as we know from the model of positive illusions (Brown & Taylor), then we should expect positive outcomes. However, huge discrepancies between perception and actual performance may indicate either an ego-inflation (when the perception is much higher than the real result) or an ego–deflation (when the perception is much lower than the real performance). The long-term expectation is that one's perception and his or her actual results should become closer. The process should also increase the chances of objective self-evaluation, both immediately and in future performances.
Self-appraisal after a test can be carried out in three different stages: before one takes the test, in order to approximate level of readiness for the test and future performance; after the test, in order to self-approximate past performance; after the test in order to juxtapose a teacher's feedback. The self-correcting process and the complete feedback have the function of validating or invalidating the selfappraisal, and it may improve future fairness. Ideally, all these stages are used, though I only propose to students the first stage; it is their choice to go through it or not.
What's more, the procedure of self-appraisal, validated by selfcorrection, might enhance the development of a healthy self-esteem grounded in actual performances and fair self-assessment. Self-appraisal of one's performance might reduce false expectations, exaggerated fears, and disappointments.
Most of the students enjoy this practice and become aware that the cognitive resources of this procedure are related to its moral challenges. The self-appraisal techniques — combined with self-correcting procedures and feedback from the teacher — can increase the self-regulatory traits of the learning process.
— Catalin Mamali
Loras College and Northeast Iowa Community College
References
- Brown, J.D. and S.E. Taylor (1988). Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin, 103, 193-210.
- James, W. (1983). The principles of psychology (F. Burkhardt, Ed.). Cambridge, MA: Harvard University Press. (Original work published 1890).
- Jung, C. G. (1977). The development of personality (H. Read, M. Fordham, G. Adler, W. McGuire, Eds., and translated by R. F. C. Hull, trans.). Bollingen Series, Volume 17. New York: Princeton University Press. (Original work published 1934).
- Jung, G. C. (1977). Two essays on analytical psychology (H. Read, M. Fordham, G. Adler, W. McGuire, Eds.; R. F. C. Hull). Bollingen Series, Volume 17. New York: Princeton University Press (Original work published 1943).
