Editorial Note: This post is from John Horgan who writes for Scientific American. The original is here. There is one change – the image used. JH’s posts are always worth reading. In this he takes on a worrying trend to regard science as sacrosanct. A recent front cover for National Geographic brought this home to me. To question whether vaccines might cause autism it seems is pretty well to be a Flat Earther. This has clear and worrying implications for me, for RxISK and for anyone who has ever been injured by a drug.
Years ago I was blathering to a science-writing class at Columbia Journalism School about the complexities of covering psychiatric drugs when a student, who as I recall had a medical degree, raised his hand. He said he didn’t understand what the big deal was; I should just report “the facts” that drug researchers reported in peer-reviewed journals.
I was so flabbergasted by his naivete that I just stared at him, trying to figure out how to respond politely. I had a similar reaction when I spotted the headline of a recent essay by journalist Chris Mooney: “This Is Why You Have No Business Challenging Scientific Experts.”
(Journalist Chris Mooney argues that the views of anti-vaccine activist Jenny McCarthy can be dismissed because she is not a “scientific expert,” but by this logic the views of journalists like Mooney should also be discounted).
Mooney is distressed, rightly so, that many people reject the scientific consensus on human-induced global-warming, the safety of vaccines, the viral cause of AIDS, the evolution of species. But Mooney’s proposed solution, which calls for non-scientists to yield to the opinion of “experts,” is far too drastic.
In support of his position, Mooney cites Are We All Scientific Experts Now?, a book by sociologist of science Harry Collins. Rejecting the hard-core postmodern view of science as just one of many modes of knowledge, Collins argues that scientific expertise is uniquely authoritative. Here’s how Mooney puts it:
“Collins carefully delineates between different types of claims to knowledge. And in the process, he rescues the idea that there’s something very special about being a member of an expert, scientific community, which cannot be duplicated by people like vaccine critic Jenny McCarthy… Read all the online stuff you want, Collins argues—or even read the professional scientific literature from the perspective of an outsider or amateur. You’ll absorb a lot of information, but you’ll still never have what he terms ‘interactional expertise,’ which is the sort of expertise developed by getting to know a community of scientists intimately, and getting a feeling for what they think. ‘If you get your information only from the journals, you can’t tell whether a paper is being taken seriously by the scientific community or not,’ says Collins. ‘You cannot get a good picture of what is going on in science from the literature,’ he continues. And of course, biased and ideological Internet commentaries on that literature are more dangerous still. That’s why we can’t listen to climate change skeptics or creationists. It’s why vaccine deniers don’t have a leg to stand on.”
Mooney is hardly the only person insisting “You Have No Business Challenging Scientific Experts.” Versions of this assertion constantly pop up in debates over hot-button scientific issues. Defenders of supposedly canonical views of global-warming, genetically modified foods and vaccines dismiss non-expert dissidents. Just last week, a friend and fellow journalist mocked meteorologists who doubt climate change–because they’re meteorologists, not climate scientists.
The irony is that the “No Business Challenging Scientific Experts” argument applies not only to activists like Jenny McCarthy but also to journalists like Mooney and me. After all, we journalists are “outsiders” and “amateurs,” especially compared to the scientists whose work we cover, so how dare we second-guess them?
I agree with Mooney and Collins on some fundamental issues. I’m not a Kuhn-style postmodernist, the kind who puts scare quotes around “truth” and “knowledge.” Science is a uniquely potent method for discovering how nature works, and it gets some things right, once and for all: the atomic theory of matter, the (basic) big bang theory, evolution by natural selection, DNA-based genetics.
Also, I give great weight to consensus and credentials, which provide a fast and dirty way to decide whether a claim should be taken seriously. One of the reasons I doubted that “cold fusion” had been achieved in the late 1980s was that scientists claiming to have observed room-temperature fusion tended to be at second-rate institutions; scientists at top-tier institutions could not replicate the results.
But the history of science suggests—and my own 32 years of experience reporting confirms—that even the most accomplished scientists at the most prestigious institutions often make claims that turn out to be erroneous or exaggerated.
Scientists succumb to groupthink, political pressures and other pitfalls. More than a half century ago, Freudian psychoanalysis was a dominant theory of and therapy for mental disorders. The new consensus is that mental illnesses are chemical disorders that need to be chemically treated.
This paradigm shift says more about the financial clout of the pharmaceutical industry–and its control over the conduct and publishing of clinical trials–than it does about the actual merits of antidepressants and other drugs. That’s why I was so stunned when that Columbia student said peer-reviewed “facts” could speak for themselves.
Here’s another example related to the work of Harry Collins, who inspired Mooney’s column. Collins’s respect for scientific expertise stems in part from his decade-long immersion in the field of gravitational-wave studies. Gravitational waves made headlines a year ago, when astrophysicists overseeing an experiment called Background Imaging of Cosmic Extragalactic Polarization 2 announced they had discovered the “first direct evidence” of inflation, a 35-year-old theory of cosmic creation. According to the group, gravitational waves triggered by inflation had distorted the big bang’s microwave afterglow in measurable ways.
No less an authority than Stephen Hawking declared that the BICEP2 results represented a “confirmation of inflation.” I nonetheless second-guessed Hawking and the BICEP2 experts, reiterating my long-standing doubts about inflation. Guess what? Hawking and the BICEP2 team turned out to be wrong.
I’m not bragging. Okay, maybe I am, a little. But my point is that I was doing what journalists are supposed to do: question claims even if–especially if—they come from authoritative sources. A journalist who doesn’t do that isn’t a journalist. He’s a public-relations flak, helping scientists peddle their products.
And it’s precisely because we journalists are “outsiders” that we can sometimes judge a field more objectively than insiders. Mooney surely agrees with me on this. There is an enormous contradiction buried within his “No Business Challenging Scientific Experts” argument. He obviously doesn’t want us to yield to every scientific consensus, only to those that he, Mooney, deems credible.
Google is reportedly working on algorithms for evaluating the credibility of websites based on their factual content. But there will never be a foolproof way to determine a priori whether a given scientific consensus is correct or not. You have to do the hard work of digging into it and weighing its pros and cons. And anybody can do that, including me, Mooney and even Jenny McCarthy.
By the way, I think McCarthy grossly overstates the dangers of vaccines–I’m glad my kids got vaccinated–but I, too, have concerns about some vaccines.
Editorial Note: On the vaccine issue, here is a recent article of interest on Vaccine Assay Secrecy by Matthew Herder and Colleagues – Herder Vaccine assay secrecy.
(This is a follow up post from John Horgan on the topic – the original is here.)
I recently knocked science journalist Chris Mooney for asserting that “You Have No Business Challenging Scientific Experts.” Non-experts have the right and even the duty, I retorted, to question scientific experts, who often get things wrong.
Far from reconsidering his stance, Mooney doubles down on it in a Washington Post column, “The science of why you really should listen to science and experts,” that defends not just scientific experts but experts in general. Mooney ends up not boosting experts’ credibility but undermining his own.
He cites a study that found that judges and other lawyers show less ideological bias—or “identity-protective cognition”–in their application of the law than law students and lay people. Titled “Ideology’ or ‘Situation Sense’? An Experimental Investigation of Motivated Reasoning and Professional Judgment,” the study was carried out by Yale law and psychology professor Dan Kahan and five other scholars.
To my mind, the study merely shows that lawyers and judges know the law better than law students and non-lawyers. That’s reassuring, but surely it does not mean we should always trust lawyers’ legal advice, especially since lawyers so often disagree on interpretations of the law. Consider the rancor of recent debates on health care, immigration, taxes, the environment and other issues in Washington, where more than one third of current Representatives and one half of Senators have law degrees, according to The National Law Journal.
Mooney nonetheless insists that the Kahan study “fits nicely alongside a growing trend toward robustly defending and reaffirming the importance of experts.” As an example of this trend, he cites the 2005 book Expert Political Judgment by political psychologist Philip Tetlock.
Mooney’s citation of Tetlock is bizarre, because Expert Political Judgment—far from a defense of experts—is a devastating critique of them. Tetlock reports on his long-term study of 284 professional pundits, including academics, government officials and journalists, who comment on politics and related issues in scholarly journals and conferences and via mass media. Over two decades, Tetlock recorded some 28,000 predictions by the experts related to wars, elections, economic collapses and other events.
In a terrific 2005 review, “Everybody’s An Expert,” New Yorker writer Louis Menand summarizes Tetlock’s conclusions as follows:
“people who make prediction their business—people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables—are no better than the rest of us. When they’re wrong, they’re rarely held accountable, and they rarely admit it, either. They insist that they were just off on timing, or blindsided by an improbable event, or almost right, or wrong for the right reasons. They have the same repertoire of self-justifications that everyone has, and are no more inclined than anyone else to revise their beliefs about the way the world works, or ought to work, just because they made a mistake. No one is paying you for your gratuitous opinions about other people, but the experts are being paid, and Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote.”
Menand’s review is loaded with gleeful one-liners, including this one: “Human beings who spend their lives studying the state of the world, in other words, are poorer forecasters than dart-throwing monkeys.” And yet this is no laughing matter. Consider how “experts” in the government, academia and media helped enable the catastrophic U.S. wars in Afghanistan and Iraq and the economic collapse of 2008. Example: New York Times columnist Thomas Friedman, who just before the U.S. invasion of Iraq expressed the hope that it would lead to “a more accountable, progressive and democratizing regime.”
How can Mooney possibly interpret Tetlock’s book as a defense of experts? Here’s how. He seizes on Tetlock’s finding that some experts were better forecasters than others. They tended to be not what Tetlock calls “hedgehogs,” who explain the world in terms of one big unified theory, but “foxes.” Foxes, Tetlock explains, “are skeptical of grand schemes,” and “diffident about their own forecasting prowess.”
In other words, the most credible experts are those who, implicitly, warn us to be wary of experts. Mooney is oblivious to this irony. “So experts really do exist,” he blithely concludes, “and they really are different from non-experts. Now, all we have to do is listen to them.”
I prefer Menand’s conclusion. He writes that “the best lesson of Tetlock’s book may be the one that he seems most reluctant to draw: Think for yourself.”
Just to be clear–my argument wasn’t that meteorologists lack the ability (or worse, the right) to challenge climate studies. My argument is that they suffer from a false sense of expertise that makes them think they can speak with authority without bothering to really understand the other field. It’s a lazy kind of arrogance. This often happens when scientists get taken with their own brilliance and think, hey, I know about genes, I’ll bet I really understand consciousness or solar energy or alien life or whatever. Of course almost everybody with a healthy ego does this to some extent, fantasizing that they are experts at something they know nothing about. But with the meteorologists (as a case study) there is a more specific type of confusion, and a more specific type of unearned claim to authority. I liked our quick Twitter exchange about the role of curiosity vs. gall. Both of them are about questioning everything, including expert testimony & peer-reviewed truths. I take a bit longer to reach full boil when I come across BS, but I get there eventually.
In American political culture, liberal commentators and advocacy journalists tend to put scientists on a sacred pedestal and are often funded to do so and attract audiences by doing so.
Climate scientists especially are not only portrayed as innocent priests and powerful seers but also as vulnerable martyrs that must be protected and defended against any criticism, even when such criticism comes from social scientists or specialist journalists who are speaking from the perspective of their own expertise.
On complex, wicked problems like climate change, the only way we identify paths forward and opportunities for political cooperation is through healthy disagreement. Criticism helps widen the menu of options that might be pursued and calls attention to faulty assumptions.
Another example of the need for informed criticism of experts, as I discussed in a recent co-authored paper, is the work of the journalist Gary Taubes who helped spur scientists to reconsider their assumptions about the linkages between diet, obesity, and other negative health outcomes.
Gorski critiques my column in a post titled “On the ‘right’ to challenge a medical or scientific consensus.” Like Mooney–and like me!–Gorski has concerns about the propagation of pseudoscience, but his piece is one long exercise in begging the question. That is, he implicitly assumes what he is attempting to argue. He writes: “It’s… important to remember that there are scientific consensuses and then there are scientific consensuses. What I mean is that some consensuses are stronger than others, something Horgan seems to ignore or downplay.” The primary point of my piece is that there is often no way to know whether a consensus is legitimate or not; scientists often claim more certainty than is merited, and hence outsiders are justified in questioning scientists’ proclamations. Gorski’s own field, oncology, offers an excellent example of this problem. For decades, the consensus was that cancer should be combatted with frequent testing and aggressive treatment, but now that consensus is unraveling. “In the end,” Gorski concludes, what Horgan seems to be arguing is that we should take pseudo-expertise seriously.” Actually, I am arguing that the public should be wary not only of pseudoscientific charlatans peddling homeopathy and “energy healing” (Gorski’s examples) but also of genuine experts like Gorski.
See also my next post, “Sociologist Steve Fuller: Scientists Aren’t More Rational Than the Rest of Us”; and my Bloggingheads.tv chat with anonymous blogger “Neuroskeptic,” an expert who displays admirable skepticism toward his own field.