Proponents of the #ExxonKnew conspiracy theory—the driving force behind nine (and counting) multi-billion-dollar lawsuits filed by U.S. cities against the world’s leading oil companies—have had an awful month.
Last week, U.S. District Judge William Alsup, who is presiding over the suits filed by San Francisco and Oakland, accused the plaintiffs’ attorneys of misleading the court during a five-hour “tutorial” on climate change when they claimed that Royal Dutch Shell, BP, ConocoPhillips, Chevron, and Exxon Mobil conspired to suppress climate-change science.
At the conclusion of the unprecedented hearing, which some had hoped would “build a foundation for future litigation,” Judge Alsup “dismissed the claim” that such a conspiracy ever occurred, effectively gutting the very foundation of the plaintiffs’ current and future lawsuits.
But that wasn’t the worst of it.
On March 1, Exxon filed papers in a Texas state court that eviscerated a 2017 “study” by La Jolla conference organizer Naomi Oreskes that sought to prove through “content analysis” that Exxon misled the public about climate change in its public statements from 1977 to 2014. In their filing, Exxon described “that ‘study’ … as a biased, results-driven endeavor to provide the patina of academic legitimacy to a corrupt enterprise.”
At the heart of Exxon’s court document was a 20-page report by Kimberly Neuendorf—a world-renowned content analysis expert—which picked apart Oreskes’ paper piece by piece until there was nothing left but the activist’s obvious bias against Exxon.
The annihilation of the paper was so comprehensive that Oreskes’ co-author, Geoffrey Supran, publicly complained that Exxon used “expert-for-hire doubt mongering” to attack their paper. Unfortunately for Supran, the “expert for hire” he was referring to was Dr. Neuendorf, a preeminent scholar who literally wrote the book on content analysis.
But perhaps most damning of all, in her critique of Oreskes’ research, Neuendorf reveals that “Supran and Oreskes … have cited my methods textbook as a source of information for the methods of content analysis.” That’s right; they used Neuendorf’s methods to try to analyze Exxon’s content. They were just really bad at it.
Following are just a few tidbits from Neuendorf’s scathing critique. (Emphasis added.) If you have the time, it’s highly worth reading.
- After a detailed review of the study, its supplementary information (“SI”), and the documents Supran and Oreskes (“S&O”) analyzed for their study, I have concluded that S&O’s content analysis does not support the study’s conclusions because of a variety of fundamental errors in their analysis. S&O’s content analysis lacks reliability, validity, objectivity, generalizability, and replicability.
- To maintain objectivity, content analysis coding ought to be conducted by coders who are at arm’s-length with regard to the research. S&O’s selection of themselves as coders is inappropriate because they are not blind to the purpose of the research or independent of each other. In fact, they were as non-blind as one could imagine.
- S&O employ a coding scheme that includes bias, is complex and requires specialized expertise beyond the scheme, and instructs coders to engage in activities that are unacceptable for content analysis.
- S&O fail to disclose their rationale for selecting some of the documents in their sample, including the set of advertorials that S&O assert “misled the public.” S&O also omit essential details about their coding scheme for the content analysis. These omissions render the study non-replicable.
- They rely on a technique known as consensus measurement, used in prior work of one of the co-authors. Unlike content analysis, which is well established, consensus measurement does not appear to be a general, scientific method, but instead, a conclusion regarding consensus about climate change opinions in search of a method.
- In short, the content analysis is unreliable, invalid, biased, not generalizable, and not replicable. Accordingly, S&O provide no scientific support for either a discrepancy among ExxonMobil’s climate change communications, or a claim that ExxonMobil misled the public.
- In light of these significant errors and omissions, the conclusions reached by S&O are not sound and should not be relied upon.