Daubert Expert
Latest News & Developments
Daubert Expert Matters
One of our recent cases illustrates a common kind of junk expert testimony that is susceptible to the careful technical lawyering that we help our law-firm clients provide to their corporate/commercial clients. The alleged expert in this case was somewhat typical of the cream of professional damages experts. He is an Economics Ph.D. from a top Ivy League university and Vice President and Director of Economic Services for a Big-4 accounting firm. He was clearly "qualified," even under a fairly strict Daubert standard. If he had been allowed to take the stand the jury would have received him as a well-credentialed expert who had a pretty convincing story to tell them. Nice hair. Toothy grin and smooth delivery. A fairly typical situation.
And he had a story that is routinely admitted: An alleged intellectual property wrong committed against the plaintiff's business kept the business from becoming worth hundreds of millions of dollars, and instead caused its bankruptcy. Never mind that the firm had never made a real profit or that its product had never actually been produced. Never mind that his valuation model would not have commanded a passing grade in a sophomore-level university finance class or that his data was financial projections made by the company in efforts to raise financing for further needed R&D. Never mind that careful inspection of these financial projections showed them to be the subject of disclaimers sufficient to keep the company from being sued if they failed to produce those numbers. Now this company was going to sue someone that had allegedly kept it from making those numbers. And they were going to use the very numbers that they had disclaimed to value their loss. They and their contingent fee attorneys demanded tens of millions to settle.
The putative expert spun the company's revenue projections, excerpted from business plans and private placement documents, through an official looking "valuation model" to yield the alleged damages. His model used six years of projections of Revenues, from which he subtracted official-looking projections for Costs of Goods Sold, Gross Margins, expenses of Marketing and Customer Support, Research and Development, and General Expenses, all yielding EBITDA projections in the range of ten to twenty million dollar per year which, when reduced by the expert witness to a present value, yielded an inferred firm value of over $100 million. Other experts were ready to testify that all the remaining problems with the product were about to be fixed and that a lucrative market existed for the product. It all looked very professional.
And note that to call their prosecution of the case a success, all the strike-suit plaintiff need accomplish is to keep the defense believing that there is a nontrivial chance that the jury will hear a damage estimate of over $100 million. Because once that happens, unless there is no straight-faced story for liability at all, juries are well-known to have a tendency to impose some kind of rough justice where they average together the $100 million that the Plaintiff's expert claims as damages with whatever the defense expert estimates as damages. In short, so long as the strike suit plaintiff can anticipate that the hugely inflated damages testimony will reach the jury, the strike suit plaintiff rationally evaluates the settlement value of the claim as a small, but nontrivial, fraction of the $100 million.
Appearances aside, this testimony was pure junk. The expert had really done nothing but spin the company's ipse dixit revenue projections, excerpted from a management-produced business plan, through a funny kind of a "valuation model" to yield the alleged damages. The projected cost reductions that he subtracted from revenues were all simple percentages of revenues which resulted in all of his spreadsheet work coming down to taking a flat percentage of projected revenues as his projections for profits. This resulted in annual profit projections in the tens of millions of dollars for this company that had never made a dollar or sold one unit of its product. He calculated the present value of these cash flows at over $100 million and this became his inferred value of the failed plaintiff firm.
Our client's client filed a pretty simple but carefully aimed objection to his testimony. We showed that the expert's model was not consistent with the Daubert/Kumho progeny and that, even if plaintiff could prove liability, the maximum amount of damages consistent with Daubert were almost surely under a million dollars. And this was still junk litigation where they would have difficulty making liability. Plaintiff dismissed within a week. The simple calculus is this: The long shot at a hundred million dollars was worth litigating over. The long shot at something under a million was not. Sophisticated Daubert motions not only control the experts in such matters, they can control the whole litigation.
One of the most common missteps in the cases occurs when courts exclude good, reliable, nonscientific testimony because it fails Daubert's test for scientific testimony. The Daubert progeny provide a test for nonscientific testimony but it raises some subtle issues and unless those issues are raised it is easy for a judge to erroneously apply the Daubert scientific test, almost guaranteeing the exclusion of the nonscientific testimony.
One of our recent cases illustrates. An accountant was prepared to testify as to damages but his testimony was opposed by a motion in limine that argued that the accountant's testimony did not meet the Daubert criteria. The motion articulated Daubert's factors 1-4 and explained how the accountant's testimony failed on all four of them, dooming it to the realm of junk science and rendering the testimony inadmissible.
The attorney who had hired the expert called us, basically admitting defeat: No, he said, the expert's methods had not been tested; no, they could not find any peer-reviewed study
that used the method; no, the expert did not know the error rates; no, there was no evidence that the method was generally accepted; and yes, the expert had created his method just for this litigation.
We showed that testing and error rate analysis were scientific criteria not applicable to this expert's methods, which were learned but not scientific. And we showed that what he had done was a simple textbook example of accounting principles straight out of a popular university intermediate accounting text, so that his testimony was not only based upon peer-reviewed principles, it looked like it was generally accepted as well. We mixed in some references to meeting GAAP and FASB standards to bolster our claims of general acceptance. The testimony came in virtually in its entirety, much to the pleasure of the attorney who had called us believing that his expert was sure to be excluded.
This analysis discusses how lawyers, confronted with an opposing expert’s regression analysis, can prevail in those cases by excluding that testimony, based upon its scientific merits (or lack thereof).
In a large percentage of our cases, our clients ask us to exclude the testimony of a statistics expert, and in a large percentage of those cases the expert testimony that we move to exclude is based on a kind of statistics that is called regression analysis. Many common statistical models are simplifications of the regression model, so most of what is said here about excluding regression-based expert testimony applies to excluding expert testimony that is based on a wide range of other statistical models. We recently consulted on a contracts matter that relied on a complex regression technique and an analysis of that case shows how the requirements (statisticians call them assumptions) of the regression model can be used to dispose of litigation.
Legal View of Regression and Statistical Analysis
Properly executed regression studies apparently meet all of the Daubert criteria: they perform tests and specify the error rates associated with those tests. They are pervasively published in the peer-reviewed scientific journals of a wide range of scientific disciplines and, properly executed, are a generally accepted scientific research technique in dozens of those disciplines. Regression is widely used in a range of non-litigation settings for purely scientific purposes.
Of course, the fact that properly executed regression studies apparently meet all of the Daubert criteria makes “properly executed” the battle ground of admissibility. All too often, courts tend to admit regression-based expert testimony with a casual reference to the fact that regression is an accepted scientific technique. The Ninth Circuit recently held that an expert could omit critical explanatory variables and still have his work be considered properly executed. See Adventures of an Econometrician Lawyer: Controlling Regression-Based Litigation with Daubert and Statistical Analysis The scientific community disagrees with this Ninth Circuit holding and the balance of this analysis discusses how lawyers confronted with opposing expert’s error-ridden regression analysis can prevail in those cases by excluding that testimony based upon its scientific merits (or lack thereof). These scientific merits begin with whether the expert met the requirements of the regression model.
Lawyering Regression Analysis under Daubert
For lawyers, the central scientific point on regression analysis is that if (and only if) the regression model is properly constructed will the regression estimators have a set of desirable properties that allow statisticians and economists to perform the testing and error rate analysis that is required under Daubert for admissibility in federal courts. Symmetrically, if counsel can establish that the proffered regression model’s requirements have been substantially violated, the scientific basis of the testimony is discredited and the testimony loses evidentiary reliability.
There are two regression problems common in the cases that stem from the substantial violation of the requirements of the regression model: model misspecification and errors in the variables. Obrey is an example of model misspecification, the more complex of the two. A regression model is misspecified if the analyst has, for example, modeled termination rates as depending on age, when those termination rates could depend on computer skill.
Statisticians say that a model is “misspecified” if the true relationship between the two variables of interest is given by one equation, but the economist models the relationship using a different equation that excludes some of the important variables. Kmenta, at 391-405 (discussing model specification and econometric tests to determine if a model is misspecified); see also Judge et al., The Theory and Practice of Econometrics at 407-46 (John Wiley & Sons 1980) (providing an overview of regression model specification tests). Regression estimates from misspecified models are considered scientifically unreliable. This is an important consideration in a range of courtroom situations.
Regression studies that meet the assumptions of the appropriate regression model have a set of desirable characteristics (best, linear, unbiased (BLUE) and consistent), Kmenta at 161, that indicate that testing and error rate analysis done with them should meet the Daubert reliability standards. Tests done with parameters estimated by misspecified models or with inaccurately measured data fail statistically. Therefore, at a minimum, they fail the testing, error rate, and general acceptance criteria of Daubert. It cannot be over-emphasized that statistical analysis that fails the Daubert standard because it fails statistically should be excluded not simply because it fails to meet a technicality that the Supreme Court has imposed. Statistical analysis that fails the Daubert test for this reason should be excluded from evidence because it is wrong.
Controlling Statistics–Based Litigation with Daubert: An Example from Contract Law Regression based litigation is everywhere and often uses very sophisticated varieties of regression analysis. A recent contracts matter that relied on a complex regression technique shows how the requirements (statisticians call them assumptions) of the regression model can be used to dispose of litigation.
In this contracts matter the plaintiff proffered a damages expert, Dr. Noll, who used a specialized form of regression that he labeled “Cox Regression.” He explained that Cox Regression is used when the available data do not conform to the requirements of the standard regression model but do conform to a slightly more lax set of requirements. The expert was a Dean and full Professor at a major research university and he claimed that he was using the model in accord with the generally accepted standards of his profession. He even offered the expert opinion that his analysis satisfied Daubert.
Used properly, Cox Regression seems likely to meet the Daubert standards, but defense experts in this contract matter identified several errors in the expert’s methods, including his choice of regressors (explanatory variables), and his systematic errors in measuring the data he relied upon. However, plaintiff’s expert, Dr. Noll, had rationalizations for these errors. Explaining the errors required highly technical, graduate level statistics that would take almost any judge or lawyer beyond the limits of their understanding. That is especially true when the statistics are explained in the nomenclature of experts: standard errors, F-tests, t-tests, z-scores and p-values; one-tail and two-tail tests; consistent and inconsistent estimates and estimators, and so forth. There are intuitive ways of explaining many standard regression and statistics concepts to non-statisticians, but Cox Regression is especially complex and is possessed of few intuitive concepts. To complicate matters, a LEXIS search for “Cox Regression” yielded no hits.
But there are many ways to debunk regression within this article’s context of commanding litigation by commanding statistics. In this instance, defense counsel noticed that Cox Regression looks very much like another regression model known as the Proportional Hazards Model and that there was one reported case on the Proportional Hazards Model.
In Coates v. Johnson & Johnson, 1982 WL 285 (N.D. Ill. 1982), aff’d, 756 F.2d 524 (7th Cir. 1985), plaintiff relied upon the Proportional Hazards Model to establish an essential element of his case. The defense proffered Professor George Neumann of the University of Chicago Business School, one of the developers of the model, who outlined the failures of plaintiff’s expert to meet the requirements (assumptions) of the Proportional Hazards Model and the resultant unreliability of the plaintiff’s expert’s methods. The court excluded the testimony of the plaintiff’s expert.
Now, the list of errors condemned by Dr. Neumann and the Seventh Circuit is very similar to the list of errors made by Dr. Noll in this contracts example. But the fact that Dr. Noll made a series of errors that had been condemned by the court in Coates was only the beginning of Coates’ usefulness: Dr. Noll’s curriculum vitae indicated that when he earned his Ph.D. at the University of Iowa, Dr. Neumann, (the prevailing expert in Coates), was a senior econometrics Professor at the University of Iowa. Defense counsel was able to argue that not only did expert Noll’s Cox Regression analysis fail Daubert, it apparently would not even pass the final exam in Professor Neumann’s Econometrics class. The case settled immediately after briefing these issues.
Maximizing clients’ interests often require advocates to undertake the complex tasks of discrediting experts’ statistical or econometric models directly, but sometimes statistically informed lawyering provides easier and more effective avenues for excluding flawed regression testimony. Challenging statistical testimony by applying learned statistical analysis to Daubert issues requires an additional set of tools, but it is a highly cost-effective litigation strategy, able to control substantial litigation with a modest investment of time and expense.
Stephen Mahle is a scientifically trained lawyer who concentrates his practice in litigating Daubert and expert testimony issues for insurance companies and their outside counsel. He has a doctorate in economics, has been a finance professor at several major universities, is webmaster of daubertexpert.com, and lectures and publishes regularly on Daubert and expert testimony issues. He can be reached at smahle@daubertexpert.com, or (561) 451-8400
One of our recent cases illustrates a common kind of junk expert testimony that is susceptible to the careful technical lawyering that we help our law-firm clients provide to their corporate/commercial clients. The alleged expert in this case was somewhat typical of the cream of professional damages experts. He is an Economics Ph.D. from a top Ivy League university and Vice President and Director of Economic Services for a Big-4 accounting firm. He was clearly "qualified," even under a fairly strict Daubert standard. If he had been allowed to take the stand the jury would have received him as a well-credentialed expert who had a pretty convincing story to tell them. Nice hair. Toothy grin and smooth delivery. A fairly typical situation.
And he had a story that is routinely admitted: An alleged intellectual property wrong committed against the plaintiff's business kept the business from becoming worth hundreds of millions of dollars, and instead caused its bankruptcy. Never mind that the firm had never made a real profit or that its product had never actually been produced. Never mind that his valuation model would not have commanded a passing grade in a sophomore-level university finance class or that his data was financial projections made by the company in efforts to raise financing for further needed R&D. Never mind that careful inspection of these financial projections showed them to be the subject of disclaimers sufficient to keep the company from being sued if they failed to produce those numbers. Now this company was going to sue someone that had allegedly kept it from making those numbers. And they were going to use the very numbers that they had disclaimed to value their loss. They and their contingent fee attorneys demanded tens of millions to settle.
The putative expert spun the company's revenue projections, excerpted from business plans and private placement documents, through an official looking "valuation model" to yield the alleged damages. His model used six years of projections of Revenues, from which he subtracted official-looking projections for Costs of Goods Sold, Gross Margins, expenses of Marketing and Customer Support, Research and Development, and General Expenses, all yielding EBITDA projections in the range of ten to twenty million dollar per year which, when reduced by the expert witness to a present value, yielded an inferred firm value of over $100 million. Other experts were ready to testify that all the remaining problems with the product were about to be fixed and that a lucrative market existed for the product. It all looked very professional.
And note that to call their prosecution of the case a success, all the strike-suit plaintiff need accomplish is to keep the defense believing that there is a nontrivial chance that the jury will hear a damage estimate of over $100 million. Because once that happens, unless there is no straight-faced story for liability at all, juries are well-known to have a tendency to impose some kind of rough justice where they average together the $100 million that the Plaintiff's expert claims as damages with whatever the defense expert estimates as damages. In short, so long as the strike suit plaintiff can anticipate that the hugely inflated damages testimony will reach the jury, the strike suit plaintiff rationally evaluates the settlement value of the claim as a small, but nontrivial, fraction of the $100 million.
Appearances aside, this testimony was pure junk. The expert had really done nothing but spin the company's ipse dixit revenue projections, excerpted from a management-produced business plan, through a funny kind of a "valuation model" to yield the alleged damages. The projected cost reductions that he subtracted from revenues were all simple percentages of revenues which resulted in all of his spreadsheet work coming down to taking a flat percentage of projected revenues as his projections for profits. This resulted in annual profit projections in the tens of millions of dollars for this company that had never made a dollar or sold one unit of its product. He calculated the present value of these cash flows at over $100 million and this became his inferred value of the failed plaintiff firm.
Our client's client filed a pretty simple but carefully aimed objection to his testimony. We showed that the expert's model was not consistent with the Daubert/Kumho progeny and that, even if plaintiff could prove liability, the maximum amount of damages consistent with Daubert were almost surely under a million dollars. And this was still junk litigation where they would have difficulty making liability. Plaintiff dismissed within a week. The simple calculus is this: The long shot at a hundred million dollars was worth litigating over. The long shot at something under a million was not. Sophisticated Daubert motions not only control the experts in such matters, they can control the whole litigation.
One of the most common missteps in the cases occurs when courts exclude good, reliable, nonscientific testimony because it fails Daubert's test for scientific testimony. The Daubert progeny provide a test for nonscientific testimony but it raises some subtle issues and unless those issues are raised it is easy for a judge to erroneously apply the Daubert scientific test, almost guaranteeing the exclusion of the nonscientific testimony.
One of our recent cases illustrates. An accountant was prepared to testify as to damages but his testimony was opposed by a motion in limine that argued that the accountant's testimony did not meet the Daubert criteria. The motion articulated Daubert's factors 1-4 and explained how the accountant's testimony failed on all four of them, dooming it to the realm of junk science and rendering the testimony inadmissible.
The attorney who had hired the expert called us, basically admitting defeat: No, he said, the expert's methods had not been tested; no, they could not find any peer-reviewed study that used the method; no, the expert did not know the error rates; no, there was no evidence that the method was generally accepted; and yes, the expert had created his method just for this litigation.
We showed that testing and error rate analysis were scientific criteria not applicable to this expert's methods, which were learned but not scientific. And we showed that what he had done was a simple textbook example of accounting principles straight out of a popular university intermediate accounting text, so that his testimony was not only based upon peer-reviewed principles, it looked like it was generally accepted as well. We mixed in some references to meeting GAAP and FASB standards to bolster our claims of general acceptance. The testimony came in virtually in its entirety, much to the pleasure of the attorney who had called us believing that his expert was sure to be excluded.
This analysis discusses how lawyers, confronted with an opposing expert’s regression analysis, can prevail in those cases by excluding that testimony, based upon its scientific merits (or lack thereof).
In a large percentage of our cases, our clients ask us to exclude the testimony of a statistics expert, and in a large percentage of those cases the expert testimony that we move to exclude is based on a kind of statistics that is called regression analysis. Many common statistical models are simplifications of the regression model, so most of what is said here about excluding regression-based expert testimony applies to excluding expert testimony that is based on a wide range of other statistical models. We recently consulted on a contracts matter that relied on a complex regression technique and an analysis of that case shows how the requirements (statisticians call them assumptions) of the regression model can be used to dispose of litigation.
Legal View of Regression and Statistical Analysis
Properly executed regression studies apparently meet all of the Daubert criteria: they perform tests and specify the error rates associated with those tests. They are pervasively published in the peer-reviewed scientific journals of a wide range of scientific disciplines and, properly executed, are a generally accepted scientific research technique in dozens of those disciplines. Regression is widely used in a range of non-litigation settings for purely scientific purposes.
Of course, the fact that properly executed regression studies apparently meet all of the Daubert criteria makes “properly executed” the battle ground of admissibility. All too often, courts tend to admit regression-based expert testimony with a casual reference to the fact that regression is an accepted scientific technique. The Ninth Circuit recently held that an expert could omit critical explanatory variables and still have his work be considered properly executed. See Adventures of an Econometrician Lawyer: Controlling Regression-Based Litigation with Daubert and Statistical Analysis The scientific community disagrees with this Ninth Circuit holding and the balance of this analysis discusses how lawyers confronted with opposing expert’s error-ridden regression analysis can prevail in those cases by excluding that testimony based upon its scientific merits (or lack thereof). These scientific merits begin with whether the expert met the requirements of the regression model.
Lawyering Regression Analysis under Daubert
For lawyers, the central scientific point on regression analysis is that if (and only if) the regression model is properly constructed will the regression estimators have a set of desirable properties that allow statisticians and economists to perform the testing and error rate analysis that is required under Daubert for admissibility in federal courts. Symmetrically, if counsel can establish that the proffered regression model’s requirements have been substantially violated, the scientific basis of the testimony is discredited and the testimony loses evidentiary reliability.
There are two regression problems common in the cases that stem from the substantial violation of the requirements of the regression model: model misspecification and errors in the variables. Obrey is an example of model misspecification, the more complex of the two. A regression model is misspecified if the analyst has, for example, modeled termination rates as depending on age, when those termination rates could depend on computer skill.
Statisticians say that a model is “misspecified” if the true relationship between the two variables of interest is given by one equation, but the economist models the relationship using a different equation that excludes some of the important variables. Kmenta, at 391-405 (discussing model specification and econometric tests to determine if a model is misspecified); see also Judge et al., The Theory and Practice of Econometrics at 407-46 (John Wiley & Sons 1980) (providing an overview of regression model specification tests). Regression estimates from misspecified models are considered scientifically unreliable. This is an important consideration in a range of courtroom situations.
Regression studies that meet the assumptions of the appropriate regression model have a set of desirable characteristics (best, linear, unbiased (BLUE) and consistent), Kmenta at 161, that indicate that testing and error rate analysis done with them should meet the Daubert reliability standards. Tests done with parameters estimated by misspecified models or with inaccurately measured data fail statistically. Therefore, at a minimum, they fail the testing, error rate, and general acceptance criteria of Daubert. It cannot be over-emphasized that statistical analysis that fails the Daubert standard because it fails statistically should be excluded not simply because it fails to meet a technicality that the Supreme Court has imposed. Statistical analysis that fails the Daubert test for this reason should be excluded from evidence because it is wrong.
Controlling Statistics–Based Litigation with Daubert: An Example from Contract Law Regression based litigation is everywhere and often uses very sophisticated varieties of regression analysis. A recent contracts matter that relied on a complex regression technique shows how the requirements (statisticians call them assumptions) of the regression model can be used to dispose of litigation.
In this contracts matter the plaintiff proffered a damages expert, Dr. Noll, who used a specialized form of regression that he labeled “Cox Regression.” He explained that Cox Regression is used when the available data do not conform to the requirements of the standard regression model but do conform to a slightly more lax set of requirements. The expert was a Dean and full Professor at a major research university and he claimed that he was using the model in accord with the generally accepted standards of his profession. He even offered the expert opinion that his analysis satisfied Daubert.
Used properly, Cox Regression seems likely to meet the Daubert standards, but defense experts in this contract matter identified several errors in the expert’s methods, including his choice of regressors (explanatory variables), and his systematic errors in measuring the data he relied upon. However, plaintiff’s expert, Dr. Noll, had rationalizations for these errors. Explaining the errors required highly technical, graduate level statistics that would take almost any judge or lawyer beyond the limits of their understanding. That is especially true when the statistics are explained in the nomenclature of experts: standard errors, F-tests, t-tests, z-scores and p-values; one-tail and two-tail tests; consistent and inconsistent estimates and estimators, and so forth. There are intuitive ways of explaining many standard regression and statistics concepts to non-statisticians, but Cox Regression is especially complex and is possessed of few intuitive concepts. To complicate matters, a LEXIS search for “Cox Regression” yielded no hits.
But there are many ways to debunk regression within this article’s context of commanding litigation by commanding statistics. In this instance, defense counsel noticed that Cox Regression looks very much like another regression model known as the Proportional Hazards Model and that there was one reported case on the Proportional Hazards Model.
In Coates v. Johnson & Johnson, 1982 WL 285 (N.D. Ill. 1982), aff’d, 756 F.2d 524 (7th Cir. 1985), plaintiff relied upon the Proportional Hazards Model to establish an essential element of his case. The defense proffered Professor George Neumann of the University of Chicago Business School, one of the developers of the model, who outlined the failures of plaintiff’s expert to meet the requirements (assumptions) of the Proportional Hazards Model and the resultant unreliability of the plaintiff’s expert’s methods. The court excluded the testimony of the plaintiff’s expert.
Now, the list of errors condemned by Dr. Neumann and the Seventh Circuit is very similar to the list of errors made by Dr. Noll in this contracts example. But the fact that Dr. Noll made a series of errors that had been condemned by the court in Coates was only the beginning of Coates’ usefulness: Dr. Noll’s curriculum vitae indicated that when he earned his Ph.D. at the University of Iowa, Dr. Neumann, (the prevailing expert in Coates), was a senior econometrics Professor at the University of Iowa. Defense counsel was able to argue that not only did expert Noll’s Cox Regression analysis fail Daubert, it apparently would not even pass the final exam in Professor Neumann’s Econometrics class. The case settled immediately after briefing these issues.
Maximizing clients’ interests often require advocates to undertake the complex tasks of discrediting experts’ statistical or econometric models directly, but sometimes statistically informed lawyering provides easier and more effective avenues for excluding flawed regression testimony. Challenging statistical testimony by applying learned statistical analysis to Daubert issues requires an additional set of tools, but it is a highly cost-effective litigation strategy, able to control substantial litigation with a modest investment of time and expense.
Stephen Mahle is a scientifically trained lawyer who concentrates his practice in litigating Daubert and expert testimony issues for insurance companies and their outside counsel. He has a doctorate in economics, has been a finance professor at several major universities, is webmaster of daubertexpert.com, and lectures and publishes regularly on Daubert and expert testimony issues. He can be reached at smahle@daubertexpert.com, or (561) 451-8400