Science and Politics:
Weathering the Storm Around
There's something not quite right about a symposium on the topic of "unpopular results" taking place before a packed audience of eager scientists and students. Nevertheless, that was the situation recently at the program on "Unpopular Results: Providing Incremental Validity at the Cost of Being Rejected" presented at the annual APS meeting in Toronto.
Unpopular Results: Providing Incremental
Validity at the Price of Being Rejected
Chair: Robyn Dawes, Carnegie Mellon University
David Faust, University of Rhode Island
Debra Poole, Central Michigan University
Bruce Rind, Temple University
Robyn Dawes, Carnegie Mellon University
The theme of the program was that science should provide incremental validity, defined by chair Robyn Dawes, of Carnegie Mellon University, as "telling people things they don't already know." But doing so is not without risk: Scientists can encounter extreme resistance from segments of society and even rejection from parts of the psychological community. It was clear from the presentations that scientists who challenge assumptions about politically-charged topics should expect to be poorly treated, threatened, harassed, or ignored. It was equally clear that the prospects of such treatment should not dissuade scientists from pursuing such knowledge.
David Faust, of the University of Rhode Island, presented a talk on "When Science Corrects Common Belief and Hence Is Rejected: The Sometimes Sad Case of the Expert Witness in Psychology and Psychiatry." He related his experiences evaluating how clinical psychologists serve as expert witnesses. Juries tend to weigh heavily the evidence they receive from experts, and perhaps give undue weight to that information in making an important decision about someone else's life. One might suppose there are requirements in place to keep expert testimony limited to statements that are supported by empirical evidence; unfortunately this has not always been the case, even in the relatively recent past.
As he described it, Faust's recommendations that clinical psychologists serving as expert witnesses adopt such criteria, earned him misery, slander, and physical threats. However, in 1993, significant standards of admissibility for expert testimony were established by a Supreme Court ruling in the case of Daubert vs. Merrill-Dow pharmaceuticals. The case hinged on a reanalysis of epidemiological data that were crucial for determining whether one of Merrill-Dow's drugs put mothers at increased risk of birth defects. The reanalysis, which served as the basis of expert testimony, purported to contradict safety claims about the drug, but had not been published or subjected to peer review. The Supreme Court's ruling on the case deals extensively with legal standards of admissibility for scientific expert testimony, including of verifiability, peer review, error rates, and general acceptability within the relevant scientific community.
Initially, the Daubert criteria were applied leniently, and many clinicians displayed hostility to these requirements, believing instead that clinical experience, credentials, board certification, and use of clinical judgment are sound bases for expert testimony. The Daubert criteria make no provision for these to be considered when judging whether expert testimony is acceptable. Further, we know from the work of Meehl, Hammond, and others in judgment and decision making psychology that assessments based on intuition or experience are reliably outperformed by simple mathematical models using the same information available to the expert using nonmechanical methods to process information. Thus, there is good reason to be skeptical when experience alone is the basis for testimony that could alter someone's life dramatically.
Concluding optimistically, Faust remarked that as lawyers and judges familiarize themselves with scientific standards, the Daubert criteria are gaining wider acceptance and more consistent application.
Debra Poole, of Central Michigan University, presented on "Psychology's Two Cultures Revisited (Again and Again)." She said her interest in the history of science led her to investigate clinical beliefs and practices regarding recovered memories of childhood sexual abuse (CSA). CSA has been the focus of a great deal of controversy in the latter half of the 20th century and Poole felt it would be worthwhile to collect data on psychologists' beliefs about the issue at different points in time. Poole collected data from practicing psychologists in order to document opinion, not intending to enter the acrimonious battle over recovered memory. However, the data became controversial because they painted an unsettling picture of current practice. Fully 25 percent of the sample felt that abused clients had to acknowledge abuse for therapy to work, that they could detect abused people without the client's explicit testimony (sometimes confidently after just one session), and that they would use varied techniques to elicit memories of CSA.
In extreme cases, some clinicians believed that as many as 100 percent of their clients with specific diagnoses were victims of CSA and that they would use hypnosis and dream interpretation as just a few of the methods to encourage recall of CSA. Many clinicians were not happy with the picture presented by Poole, and a barrage of email followed the published article. Some of this verged on harassment, and even led to a lawsuit delivered to the trustees of Poole's university.
Poole says that on several occasions during the research and its aftermath, she felt as if she were experiencing two divergent cultures: that of experimental psychologists versus that of clinically-oriented psychologists. She presented data showing significant differences on surveys of the two groups, particularly regarding questions of epistemology (i.e., how we "know"). Compared to experimental psychologists, she says clinicians tend to be more open to alternative ways of knowing beyond the scientific method, and are more apt to believe intuition is valid. Poole has had difficulty publishing this data and recommended that we create an outlet for examining how psychology operates in our society.
Bruce Rind, of Temple University, discussed "The Importance of Skeptical Inquiry into Assumptions about Child Sexual Abuse, and the Costly Consequences for this Inquiry." He too spoke about being on the receiving end of backlash for inquiry into CSA. In fact, the fallout in his case has been more public and extreme than in most. Rind's research began as an investigation of the assumption that CSA is a uniquely devastating experience, even for consenting teenagers. This assumption undergirded much of the work on recovered memory and multiple personality disorder in the late 1970s-80s, two areas where science and non-science have frequently collided.
With colleagues, Rind presented a review paper that improved in several ways on earlier papers, finding this assumption to be overstated, and that while all child abuse is damaging, the nature and effects of the damage can vary depending on the age of the victim. Ideally, this conclusion could be an important advance in our understanding of the effects and potential treatment of child abuse.
But instead of entering a scholarly debate, Rind's article became the focus of a widespread and angry campaign against his work, which had passed the rigorous review board of the journal Psychological Bulletin. It all started when some fringe groups misstated Rind et al's conclusions to serve their own interests. Subsequently, these misstated interpretations were broadcast by popular radio host "Dr. Laura" Schlesinger, who condemned the Rind piece in part because she was not familiar with meta-analysis, the statistical technique used by Rind and colleagues to review earlier studies. "I've never heard of this in science," she said. (Side note: Dr. Laura must not be aware that pooling results in one form or another is how just about all progress occurs in science.)
The publicity generated by Dr. Laura and others reached members of the U.S. House of Representatives, most notably majority whip Rep. Tom Delay (R-TX). The result was censure by the House and turmoil for the journal review process. According to Rind, political pressure led to an unprecedented second round of review for an already published article. Rind says that the reverberations from this controversy are continuing, and that undue consideration of the political implications of otherwise scientifically sound articles may delay or prevent publication of those articles, an approach which Rind characterizes as antiscientific.
Dawes capped the symposium with one last tale of science and politics, "Saving Lives, Symbolic Politics, and Secondary Irrationality." He served on a committee of the prestigious National Academy of Sciences, mandated to examine the relationship of behavior to AIDS, especially behaviors involved in illicit drug use. Intravenous (IV) drug use and related behaviors are leading factors in the spread of AIDS, and it seems evident that public health programs are needed to counteract those behaviors or reduce their impact. But what seems obvious to some is not necessarily obvious to others.
For some time, noted Dawes, several groups have claimed that needle exchange programs were contradictory because drug users don't care about AIDS, and they voiced the unsubstantiated claim that needle exchange would encourage IV drug use. However, the NAS committee concluded that needle exchange programs should be implemented, based on evidence from countries that had such programs. They also recommended that effectiveness be determined empirically with behavioral surveys and randomized tests of the program.
Despite these scientific findings, a law was passed preventing federally-funded needle exchange programs, giving the President authority to make a decision on the matter without empirical evidence. Further, said Dawes, an effective lobbying and cultural machine began denouncing the committee. Dawes suggests that the people involved in this lobbying have difficulty seeing scientists as impartial observers and automatically assume they are attempting to steer society in immoral directions. Although based on misconceived views, this group successfully halted needle exchange legislation in Congress.
Meanwhile, a needle exchange program was carried out in New Haven from the back of a van. The program's administrators gave solid evidence of that the needles were being used, and a concomitant decrease in HIV infection followed. While not large, these effects were significant. Based on this evidence, there were signs that the Clinton administration would implement the program, but pressure forced the administration to back off and instead exhort counties and cities to implement the programs. In other words, the government believes there is evidence for the effectiveness of needle exchange programs in halting the spread of AIDS. But these unpopular scientific results drew protests by special interest groups that had traditionally protested the program and who successfully thwarted the implementation of scientifically-based public health policy.
Dawes drew a parallel to avoidance conditioning, in which an animal will continue an unnecessary avoidance response simply because testing whether the response is needed could result in punishment. Years later, policy on this issue hasn't changed and Dawes never has been personally attacked as his panel has been, but his fate is awful in its own way - his recommendations are simply ignored.
Although the stories shared by the panel were sometimes so outrageous as to inspire laughter, the note struck is ultimately a sad reflection on our society's conflicted feelings and lack of understanding about psychological science. The panelists did not appear embittered by their experiences, but their stories illustrate how what seems rational and justifiable to the scientist isn't always considered legitimate to non-scientists. Hopefully, this is not a cause for despair, but instead inspires redoubled efforts at scientific education in psychology across all levels of society. - Christopher J. Anderson