Winning your case by excluding statistical expert testimony
This analysis discusses how lawyers, confronted with an opposing expert’s regression analysis, can prevail in those cases by excluding that testimony, based upon its scientific merits (or lack thereof).
In a large percentage of our cases our clients ask us to exclude the testimony of a statistics expert, and in a large percentage of those cases the expert testimony that we move to exclude is based on a kind of statistics that is called regression analysis. Many common statistical models are simplifications of the regression model, so most of what is said here about excluding regression-based expert testimony applies to excluding expert testimony that is based on a wide range of other statistical models.
We recently consulted on a contracts matter that relied on a complex regression technique and an analysis of that case shows how the requirements (statisticians call them assumptions) of the regression model can be used to dispose of litigation.
Legal View of Regression and Statistical Analysis
Properly executed regression studies apparently meet all of the Daubert criteria: they perform tests and specify the error rates associated with those tests. They are pervasively published in the peer-reviewed scientific journals of a wide range of scientific disciplines and, properly executed, are a generally accepted scientific research technique in dozens of those disciplines. Regression is widely used in a range of non-litigation settings for purely scientific purposes.
Of course, the fact that properly executed regression studies apparently meet all of the Daubert criteria makes “properly executed” the battle ground of admissibility. All too often, courts tend to admit regression-based expert testimony with a casual reference to the fact that regression is an accepted scientific technique. The Ninth Circuit recently held that an expert could omit critical explanatory variables and still have his work be considered properly executed. See Adventures of an Econometrician Lawyer: Controlling Regression-Based Litigation with Daubert and Statistical Analysis The scientific community disagrees with this Ninth Circuit holding and the balance of this analysis discusses how lawyers confronted with opposing expert’s error-ridden regression analysis can prevail in those cases by excluding that testimony based upon its scientific merits (or lack thereof). These scientific merits begin with whether the expert met the requirements of the regression model.
Lawyering Regression Analysis under Daubert
For lawyers, the central scientific point on regression analysis is that if (and only if) the regression model is properly constructed will the regression estimators have a set of desirable properties that allow statisticians and economists to perform the testing and error rate analysis that is required under Daubert for admissibility in federal courts. Symmetrically, if counsel can establish that the proffered regression model’s requirements have been substantially violated, the scientific basis of the testimony is discredited and the testimony loses evidentiary reliability.
There are two regression problems common in the cases that stem from the substantial violation of the requirements of the regression model: model misspecification and errors in the variables. Obrey is an example of model misspecification, the more complex of the two. A regression model is misspecified if the analyst has, for example, modeled termination rates as depending on age, when those termination rates could depend on computer skill.
Statisticians say that a model is “misspecified” if the true relationship between the two variables of interest is given by one equation, but the economist models the relationship using a different equation that excludes some of the important variables. Kmenta, at 391-405 (discussing model specification and econometric tests to determine if a model is misspecified); see also Judge et al., The Theory and Practice of Econometrics at 407-46 (John Wiley & Sons 1980) (providing an overview of regression model specification tests). Regression estimates from misspecified models are considered scientifically unreliable. This is an important consideration in a range of courtroom situations.
Regression studies that meet the assumptions of the appropriate regression model have a set of desirable characteristics (best, linear, unbiased (BLUE) and consistent), Kmenta at 161, that indicate that testing and error rate analysis done with them should meet the Daubert reliability standards. Tests done with parameters estimated by misspecified models or with inaccurately measured data fail statistically. Therefore, at a minimum they fail the testing, error rate and general acceptance criteria of Daubert. It cannot be over-emphasized that statistical analysis that fails the Daubert standard because it fails statistically should be excluded not simply because it fails to meet a technicality that the Supreme Court has imposed. Statistical analysis that fails the Daubert test for this reason should be excluded from evidence because it is wrong.
Controlling Statistics–Based Litigation with Daubert: An Example from Contract Law
Regression based litigation is everywhere and often uses very sophisticated varieties of regression analysis. A recent contracts matter that relied on a complex regression technique shows how the requirements (statisticians call them assumptions) of the regression model can be used to dispose of litigation.
In this contracts matter the plaintiff proffered a damages expert, Dr. Noll, who used a specialized form of regression that he labeled “Cox Regression.” He explained that Cox Regression is used when the available data do not conform to the requirements of the standard regression model but do conform to a slightly more lax set of requirements. The expert was a Dean and full Professor at a major research university and he claimed that he was using the model in accord with the generally accepted standards of his profession. He even offered the expert opinion that his analysis satisfied DaubertI.
Used properly, Cox Regression seems likely to meet the DaubertI standards, but defense experts in this contract matter identified several errors in the expert’s methods, including his choice of regressors (explanatory variables), and his systematic errors in measuring the data he relied upon. However, plaintiff’s expert, Dr. Noll, had rationalizations for these errors. Explaining the errors required highly technical, graduate level statistics that would take almost any judge or lawyer beyond the limits of their understanding. That is especially true when the statistics are explained in the nomenclature of experts: standard errors, F-tests, t-tests, z-scores and p-values; one-tail and two-tail tests; consistent and inconsistent estimates and estimators, and so forth. There are intuitive ways of explaining many standard regression and statistics concepts to non-statisticians, but Cox Regression is especially complex and is possessed of few intuitive concepts. To complicate matters, a LEXIS search for “Cox Regression” yielded no hits.
But there are many ways to debunk regression within this article’s context of commanding litigation by commanding statistics. In this instance defense counsel noticed that Cox Regression looks very much like another regression model known as the Proportional Hazards Model, and that there was one reported case on the Proportional Hazards Model.
In Coates v. Johnson & Johnson, 1982 WL 285 (N.D. Ill. 1982), aff’d, 756 F.2d 524 (7th Cir. 1985), plaintiff relied upon the Proportional Hazards Model to establish an essential element of his case. The defense proffered Professor George Neumann of the University of Chicago Business School, one of the developers of the model, who outlined the failures of plaintiff’s expert to meet the requirements (assumptions) of the Proportional Hazards Model and the resultant unreliability of the plaintiff’s expert’s methods. The court excluded the testimony of the plaintiff’s expert.
Now, the list of errors condemned by Dr. Neumann and the Seventh Circuit is very similar to the list of errors made by Dr. Noll in this contracts example. But the fact that Dr. Noll made a series of errors that had been condemned by the court in Coates was only the beginning of Coates’ usefulness: Dr. Noll’s curriculum vitae indicated that when he earned his Ph.D. at the University of Iowa, Dr. Neumann, (the prevailing expert in Coates), was a senior econometrics Professor at the University of Iowa. Defense counsel was able to argue that not only did expert Noll’s Cox Regression analysis fail DaubertI, it apparently would not even pass the final exam in Professor Neumann’s Econometrics class. The case settled immediately after briefing these issues.
Maximizing clients’ interests often require advocates to undertake the complex tasks of discrediting experts’ statistical or econometric models directly, but sometimes statistically informed lawyering provides easier and more effective avenues for excluding flawed regression testimony. Challenging statistical testimony by applying learned statistical analysis to Daubert issues requires an additional set of tools, but it is a highly cost effective litigation strategy, able to control substantial litigation with a modest investment of time and expense.
Stephen Mahle is a scientifically trained lawyer who concentrates his practice in litigating Daubert and expert testimony issues for insurance companies and their outside counsel. He has a doctorate in economics, has been a finance professor at several major universities, is webmaster of daubertexpert.com, and lectures and publishes regularly on Daubert and expert testimony issues. He can be reached at firstname.lastname@example.org, or (561) 451-8400