Junk Science & U.S. Civil Juries: Roundup Trials

“The art of junk science is to brush away just enough detail to reach desired conclusions, while preserving enough to maintain an aura of authoritative science.” This statement by Peter William Huber in his 1991 book Galileo’s Revenge: Junk Science in the Courtroom was never more evident than in current civil trials involving complex scientific and medical studies and associated testifying experts. Federal and state courts have Daubert and other applicable state level standards for determining if proposed experts are qualified by education and experience to provide expert opinions. However, the qualifying process for experts does not extend to the validity and forthrightness of subsequent expert testimony for juries incapable of identifying the nuances and weighing the significance of scientifically oriented expert testimony. In addition, opening statements and closing arguments can set-up and reinforce juror messaging of junk science.

Current state and federal trials concerning the use Roundup herbicide “causing” one of the nearly 60 diverse sub-types of Non-Hodgkin’s Lymphoma (NHL) provide a judicial environment amenable for junk science. The epidemiological case or cohort studies, animal toxicology studies and genotoxicological studies of general causation establish the foundation for medical specialists to incorporate plaintiff specific medical factors before opining whether Roundup caused the plaintiff’s NHL sub-type. Further complicating the trials is the legal discovery of 20 million Monsanto historical documents. Plaintiffs allege a number of these documents indicate Monsanto engaged in obscuring, deriding, “ghostwriting” or not performing studies pertinent to Roundup or glyphosate assurance of no-cancer risk for Roundup users. Monsanto maintains the parsing of activity or process segments coupled with erroneous interpretations requires they provide witness testimony for accurate and contextual interpretations and furthermore they provided supplemental scientific information for these scientific reviews while the underlying studies remained wholly intact.

The first Roundup trial in a San Francisco state court combined causation and “document interpretation” and generated an environment laden with junk science and higher order speculation. The unique trial environment combined with the speculative leaps by plaintiff’s medical expert supported an opinion plaintiff’s rare and idiopathic NHL cancer sub-type was incurred during the first days of Roundup spraying, required a much shorter latency period than significant exposure to well recognized cancer agents and was based upon statistically significant, but not causally associated odds ratios from exploratory case studies of epidemiology. (Upon refusal of trial judge to issue a summary dismissal notwithstanding the jury verdict, the verdict was appealed)

Federal Judge Chhabria of the San Francisco District Court recognized the issues associated with intermixing causation and liability into one trial and acceded to Monsanto’s request for a bifurcated trial with a first phase based on scientific studies determining causation and an “if needed” second phase for liability. However, even within a more disciplined and focused causation phase, there is evidence of junk science. For example, testimony regarding specified genotoxological studies without regard for mode of exposure, mammalian species or not, significance of test/assay type and the importance within the overall weight of evidence is essentially junk science and is communicated to the jury. Similarly, identifying issues within one mouse study is junk science if not presented concurrently with the three other mice studies of the same breed and treatment protocols because replication confirms or denies scientific conclusions and junk science opinions can be passed directly to jurors.

In addition, during the Hardeman v. Monsanto trial in San Francisco federal court, an apparently highly regarded epidemiological analysis, the Zhang 2019 meta-analysis of several epidemiology studies was evaluated and touted by two of plaintiff’s expert witnesses, an epidemiologist and a pathologist. The study projected a relative 41% overall increase of NHL cancer among highly exposed Roundup users such as that noted by plaintiff’s anecdotal use.

However. the study was not well-received by defense’s expert witness, Lorelei Mucci, cancer epidemiologist, Associate Professor of Harvard University (among many other roles) who had a different assessment of Zhang analysis and indicated a variety of deficiencies sufficient for characterizing the study as junk science during direct examination (trial transcript):

Q. Okay. So just on meta-analysis, are you familiar with a paper called Zhang that was published within the past few weeks? A. Yes, I am. Q. Have you reviewed that paper? A. Yes, I have. Q. What is your opinion of it? A. So the method that they used to combine in the meta-analysis the six studies was a flawed approach. There are many articles written about the appropriate way to do meta-analysis when you’re looking at different levels of exposure. And I can describe it as in this study they mixed -­three of the studies only had ever versus never exposure, two of the studies had very limited estimates of dose-response and then they had the AHS. So when you’re doing that, you’re mixing basically apples and oranges and pineapples all together, and it’s just not a valid approach to doing meta-analyses. Q. And let me ask you specifically about AHS, Andreotti, 50,000-plus person study. Did Zhang include all of AHS? A. No, they did not. In fact, they only used a very small slice, less than 20 percent of the full data in that study. And the importance of that is that the full power of the AHS was underestimated in that meta-analysis. I think the other part about the meta-analysis to mention is that all of those case-control studies that did not do a proper adjustment for pesticide use, you’re putting those odds ratios into the meta-analysis. So if you put flawed data into a meta-analysis, you’re going to get flawed data out of the meta-analysis. Q. So, for instance, De Roos, is De Roos part of Zhang? A. Yes, it was. Q. Okay. And so if De Roos is 35- to 40-year-old data before you put it in Zhang, is it still 35- to 40-year-old data once it’s in Zhang? A. Yes, absolutely.

Subsequently, Dr. Mucci was cross-examined by the plaintiff’s lawyer:

Q. Okay. And so this one is a little different than the other three, right? A. Yes, it is. Q. And this one is a little different because they actually did sort of a dose response how to use meta-analysis, right? A. No. Actually, that’s not really correct. In fact, what they did was sort of a hodgepodge of results. Three of the six studies did ever-never. Two of the remaining studies had some measure of dose response using not adjusted for pesticides. And then there was one clear dose response. So it wasn’t clearly a dose response analysis. Q. Okay. So — so the Zhang authors looked at the high use — the high-exposure people, right? A. No. Again, just to be clear, three of the six studies didn’t have dose response to present. So in those they were only looking at ever versus never exposure. Q. Okay. But some of them — some of the analysis they did were with respect to the high-exposure people; is that fair? A. Well, the highest in those particular studies, yes. Q. Sure. A. But, again, just to be clear, two of those three studies that had information on dose response were not adjusted for other pesticides and that’s an important factor in any meta-analysis. Q. Sure. Okay. So unlike — and the three previous meta-analyses — Chang, Schinasi and IARC — did not do those high-exposure analyses, right? A. No. They focused on ever versus never exposure.Q. So this was sort of a new analysis done, right? A. It was new, but it was not a correct methodology. This isn’t an appropriate way of combining results from studies. It is not appropriate to mix the results from different categories of exposure. We don’t do this in epidemiology. Q. Okay. And so what the Zhang authors found was — and it says, Accepted Manuscript across here. I’m sorry if that is a little confusing to read. I will highlight it. But what the Zhang authors found was, Overall in accordance with the evidence from experimental animals and mechanistic studies, our current meta-analysis of human epidemiological studies suggests a compelling link between exposures to glyphosate based-herbicide — and Roundup is a glyphosate based-herbicide, right? A. Yes, it is. Q. Okay. An increased risk for NHL, right? A. That is what it said. However, what they have come up with is a biased analysis combining data including results that were not adjusted for other pesticides. And let me be clear, as an epidemiologist when we review the studies, we review the independent epidemiological literature. We look at the quality of the data going into those studies. And as I discussed earlier, there were many flaws, including the fact that they were not adjusted properly for other pesticides. So I think when you take into account the results of a meta-analysis, you have to think about what was the quality of the studies going into it.

The Zhang meta-analysis clearly meets and exceeds the criteria for junk science of Huber; i.e., “The art of junk science is to brush away just enough detail to reach desired conclusions, while preserving enough to maintain an aura of authoritative science.” Why would the plaintiff present this study during opening, have two experts praise during plaintiff’s case, attempt to revive during defense’s case and again use within closing when it is such a clear example of junk science. The answer involves the quirky nature of trials with juries unknowing in the relevance and significance of testimony regarding scientific studies. Plaintiff’s case is presented first and defense lawyers can cross-examine, but clear impeachment can only be made where there is a documented foundation in the current trial and the defense expert will not testify for several days. Alternatively, the defense can also use the defense expert to impugn not only the credibility of the study, but the plaintiff(s) experts as well. Whether the study will positively or negatively affect the jury is largely unknown and the plaintiff’s decision to present depends more upon trial and jury dynamics than concern for scientific or relevant truths.

There were no remedies available for Judge Chhabria because he or any other judge does not have advance detailed knowledge of a recently published meta-analysis or plaintiff’s intended use for bolstering a case weak on epidemiological evidence, but loaded with plaintiff’s anecdotal evidence for extraordinary exposure to Roundup through spraying activities. Neither can he determine the predicted effect on unsophisticated jurors, although he can recognize a substantial cumulative amount of trial time has been spent on a study which has no scientific relevance for the jury.

Judge Chhabria carefully precluded the more egregious examples of speculation noted in the San Francisco state court trial by requiring the plaintiff to simultaneously denote whether epidemiological results were statistically significant, but not adjusted for confounding variables such as the use of other pesticides. However, there are studies which are left to individual judicial judgement in the absence of guidelines or precedents. For example, the De Roos 2003 epidemiology study played a pivotal role during causal phase of trial because one result was statistically significant and adjusted for confounding variables. The defense maintained the result was incorrect for highly technical reasons associated with “data sparcity” as well as inherent biases within the study and a subsequent pooled analysis of demonstrated there were no causal associations within the study data. Judge Chhabria left this judgement to the jury and appellate courts will subsequently determine if a precedent is applicable or needed for these types of instances. (Jury found for plaintiff and Roundup being a significant factor for Hardeman’s NHL cancer.)

Another unique aspect of the Roundup trials is the more than 800 scientific studies (excluding the innumerable number of studies relevant to causal or other medical factors) pertinent to glyphosate and glyphosate formulations such as Roundup which have been evaluated by International Agency for Cancer Research (IARC), U.S. Environmental Protection Agency as well as other key regulatory and scientific agencies world-wide. In fact, Roundup and other glyphosate formulations and associated container labels can only be sold with the approval of these regulatory agencies. The plaintiff maintained the IARC conclusion of “probable carcinogen” was preeminent during the state court trial. The defense insisted the IARC conclusion used an undisciplined analytical approach for hazard analysis in lieu of a structured “weight of evidence” methodology and risk analysis which is the only appropriate approach for determining human exposure risks and is also the rationale for its use by all regulatory/scientific agencies.

Judge Chhabria noted the state court trial focused on the conclusions of IARC, EPA and other regulatory agencies and distracted jurors from the underlying studies, the most important for juror decisions, and implemented trial guidelines accordingly which limited references to overall conclusions.

Is there a remedy for junk science which can provide reasoned verdicts for complex civil cases? Beyond the inherent unfairness, evidentiary and legal issues derived from junk science provide the basis for appeals and appellate courts are left with the resolution in one fashion or another in a highly inefficient utilization of judicial resources for the federal as well as state judicial systems. Possibly, but not plausibly, there are changes in laws which could could further restrict junk science testimony for federal and individual state judiciaries; i.e., don’t wait with bated breath.

Ideally, bench trials upon request by defense as used by all other nations other than the U.S. in the tradition of English common law would eliminate much of the junk science, greatly improve judicial economy and provide a written verdict with explicit reasoning for review by interested parties. (Generally, a majority of countries are inheritors of Napoleonic legal codes which never use juries.) Again, statutory authority would be required federally and individually for each state and the probability of this being accomplished is non-existent.

Unfortunately, the most optimistic prognosis indicates complex civil cases with varying degrees of junk science will continue to clog all levels of federal and state judiciary systems and incremental progress will only occur through establishment of incremental legal precedents.