Comments on Proposed Federal Rule of Evidence 707 and Amendments to Rule 609

SUMMARY

The Federal Courts Committee (Jonathan New, Chair) submitted a letter to the U.S. Judicial Conference Committee on Rules of Practice and Procedure addressing proposed Rule 707 and an amendment to Rule 609 of the Federal Rules of Evidence. The Committee supports proposed Rule 707, which would permit courts to evaluate the reliability of machine-generated evidence such as evidence produced by artificial intelligence before admission unless accompanied by expert testimony. The letter explains that AI-generated evidence presents unique reliability concerns, including opacity, evolving technology, and risks of error or manipulation, and argues that existing evidentiary rules do not adequately safeguard against these dangers. A pretrial judicial gatekeeping process, the Committee argues, is necessary to prevent unreliable machine-generated evidence from misleading juries and undermining fairness.

The Committee also supports the proposed amendment to Rule 609(a)(1), which governs the use of prior felony convictions to impeach a defendant’s credibility. The letter explains that, despite the rule’s original intent to protect defendants from unfair prejudice, courts have frequently admitted highly prejudicial prior convictions with minimal relevance to truthfulness, discouraging defendants from testifying and increasing the risk of wrongful convictions particularly for people of color. By requiring that the probative value of a prior conviction substantially outweigh its prejudicial effect, the proposed amendment would restore the rule’s protective purpose and promote fairer trials.

REPORT

Advisory Committee on Evidence Rules
U.S. Judicial Conference Committee
on Rules of Practice and Procedure
Thurgood Marshall Federal Judiciary Building
1 Columbus Circle, NE
Washington D.C. 20002

Re: Comment on Proposed New Rule 707 and Amendment to Rule 609

Dear Advisory Committee Members:

On behalf of the Federal Courts Committee of the New York City Bar Association (“City Bar”),[1] we respectfully submit the following comments on proposed Rule 707 and the proposed amendment to Rule 609 of the Federal Rules of Evidence. The City Bar greatly appreciates the opportunity for public comment on the proposed amendments provided by the Judicial Conference’s Committee on Rules of Practice and Procedure.

I. PROPOSED RULE 707

The Advisory Committee has proposed a new rule to govern the reliability inquiry for machine-generated evidence. The City Bar supports adoption of rules governing machine-generated evidence; however, we strongly recommend revisions to make more explicit the issues targeted by the evidence rule and that no evidence rule be adopted unless accompanied by corresponding revisions to Civil Rule 26 and Criminal Rule 16 to incorporate essential notice elements.

1. Considerations

a. Expert Testimony

The Federal Courts Committee agrees that the time is ripe to consider a rule addressing the admissibility of machine-generated evidence, and particularly evidence derived from machine learning. Given both the increasing complexity of such evidence and the danger that it will be uncritically accepted by a finder of fact, there is a need to establish guidelines for its admissibility. As currently drafted, it appears that Rule 707 will effectively require machine-generated evidence to be admitted through expert testimony except when its reliability is unchallenged or is susceptible to judicial notice. It is unclear whether any lay witness will be able to provide the testimony necessary to establish reliability under Rule 707. Under these circumstances, the note should explicitly indicate the preference for expert testimony. We recognize that proposed Rule 707 would be the first rule to effectively require expert reliability showings for a specific type of evidence. But the uniquely challenging nature of machine learning warrants this innovation.

Although it is unclear how often the rule will be used, any rule permitting lay testimony regarding such evidence should more specifically identify the type of evidence targeted and require discovery disclosures regarding such evidence. We discuss each of these recommendations below.

b. Machine-Generated Inferential Evidence

The underlying concern animating Proposed Rule 707 is not machine involvement, but machine-generated inferences that substitute for expert reasoning and whose reliability is not self-evident. As drafted, however, the rule focuses on “machine-generated evidence” broadly, forcing such evidence into an expert-testimony framework without squarely addressing in the Rule’s language the core issue: the accuracy and reliability of inferential outputs.

Nearly all modern evidence is machine-generated in some sense, including spreadsheets, GPS data, metadata, timestamps, calculator outputs, digital images, and sensor readings. Many of these outputs result from deterministic, well-validated processes that resemble measurement rather than expert analysis. Proposed Rule 707 is not aimed at these routine outputs, but at situations where a machine effectively performs analytical reasoning. For this reason, “machine-generated” is an overbroad category. It describes the source of the output, not the nature of the reasoning behind it.[2]

Although the draft does not make this distinction explicitly, it appears to rest on an implicit divide between machines that merely measure and machines that generate inferences. Measurement-based outputs are generally less likely to require expert testimony because they result from deterministic and transparent processes governed by calibration and standard operation. Errors associated with such outputs tend to be mechanical rather than methodological. Examples typically falling into this category include thermometers, radar guns, GPS coordinates, raw surveillance video, and basic spreadsheet arithmetic, all of which are usually admissible without expert explanation when their accuracy is not contested.

Machine-generated inferential evidence refers to outputs produced by systems that interpret data or draw conclusions using statistical, model-based, or probabilistic techniques, or that otherwise engage in processes that assess, predict, classify, or generate conclusions in a manner analogous to expert reasoning. Such systems are unlike machines that simply record information or perform straightforward calculations. Examples of such inferential outputs include probabilistic genotyping (a form of DNA analysis), facial or voice recognition results, AI-based image or audio reconstruction, predictive analytics, and conclusions generated by generative artificial intelligence systems. In contrast to measurement-based outputs, these do not result from methods that are established, documented, and transparent. Because the reliability of these outputs cannot be readily evaluated by lay factfinders, they present distinct admissibility concerns that are not addressed by traditional approaches to machine-generated measurements.

The Federal Courts Committee therefore recommends changing the language of the rule from “machine-generated evidence” to “machine-generated inferential evidence” to capture this distinction. The proposed rule would be more likely to permit admission of machine-generated measurements when their accuracy is not disputed. Inferential outputs would also be more likely to be admitted when they are supported by expert testimony that explains the underlying methodology, assumptions, validation, and limitations of the system. And such a rule would also tend to promote admission where the reliability of the inferential output is well established, error rates are understood, and the opposing party does not meaningfully contest the evidence’s accuracy.

Conversely, the revised rule would have the effect of discouraging admission when inferential machine outputs are offered without expert testimony, particularly where those outputs consist of generative AI conclusions offered for their truth, such as legal or factual conclusions produced by a large language model. Outputs generated by opaque systems with no validation, unknown error rates, or a demonstrable risk of hallucination or fabrication also raise heightened concerns. Similarly, evidence presented as “mere computer output” or characterized as a business record in an effort to avoid Daubert scrutiny is unlikely to satisfy the reliability demands that underlie the proposed rule.

A shift to the “inferential evidence” language discussed above will also avoid the need to carve out the “output of simple scientific instruments” from the rule in the current proposal. That distinction is unclear when applied to the output of computers, computer software, web-based services, and the like, which are not simple scientific instruments but can produce both non-inferential and inferential outputs. To aid courts and litigants, we further propose that the committee note explain the distinction between inferential and non-inferential evidence covered by the rule (if this distinction is adopted).

c. Mandatory Notice Provisions

The Advisory Committee Note to the proposed rule provides that expert notice principles “should” apply to disclosures under the new rule. The Federal Courts Committee strongly recommends instead revisions to Civil Rule 26 and Criminal Rule 16 to incorporate Rule 707 into regular discovery mechanisms. At bottom, if machine-generated evidence is treated like expert evidence, then corresponding rule changes are necessary to ensure sufficient and timely disclosures to enable parties to determine whether to contest the material. In our view, revisions to the notice rules are the more important reform for litigants to ensure the functioning of the adversarial process in this new area of evidence.

Although some machine-generated evidence can be produced faster than a typical expert report, sometimes even in a matter of minutes, sufficient time and disclosures will be critical to ensure that parties may obtain discovery into the underlying algorithms and data that yielded the evidence. We recommend revisions to both Civil Rule 26 and Criminal Rule 16 that require disclosure of the machine-generated inferential evidence a party may use at trial and the facts or data considered in forming the machine-generated evidence. In addition, in the civil context, a party should have the opportunity to depose anyone identified as a witness who will offer evidence under Rule 707. Rule 707-related disclosures may include a witness or the court examining the underlying algorithms, training data, error rates, and other computational statistics that may be fairly complex. Discovery on these issues should be conducted alongside expert discovery.

d. Establishing Reliability

Many questions will remain for the courts to tackle even if some form of Rule 707 is promulgated, as discussed above. For example, since large language models are probabilistic, they will not provide the same response to the same prompt every time. Will this be viewed as inherently unreliable, or will courts accept some level of uncertainty in defining reliability? At what point will a machine be considered sufficiently reliable that a court may take judicial notice such that expert testimony is unnecessary? These questions and others like them are significant and the answers will affect the outcome of specific admissibility determinations. They do not, however, undermine the value of a rule that focuses the parties and the court on the need to establish the reliability of machine-generated evidence and provides advance notice to the other parties of its proposed admission at trial.

2. Proposed Revisions

The Federal Courts Committee recommends the following revisions to the proposed Rule:

Evidence Rule 707. Machine-Generated Inferential Evidence

When machine-generated inferential evidence is offered without an expert witness and would be subject to Rule 702 if testified to by a witness, the court may admit the evidence only if it satisfies the requirements of Rule 702(a)–(d). This rule does not apply to the output of simple scientific instruments.

We also recommend that the Advisory Committee’s note include a statement that: Machine-generated inferential evidence should principally be sought to be introduced through expert testimony and its introduction without such testimony should be limited to circumstances when the evidence is sufficiently self-explanatory and non-controversial not to require expert framing.

We further suggest that the note include an explanation of the scope of the “inferential” evidence covered by the rule, excluding not only the output of simple scientific instruments, but also computer-generated outputs in which the computer is only providing measurement or other more deterministic outputs, as opposed to outputs in which the computer is contributing inferential or artificially intelligent analysis.

II. RULE 609

The Federal Courts Committee supports the proposed amendment to Federal Rule of Evidence 609(a)(1). As currently applied by courts, Rule 609(a)(1) imposes an unwarranted burden on criminal defendants’ right to testify and, ultimately, to a fair trial. The proposal to raise the standard that must be satisfied before the admission of a prior conviction for the purpose of impeaching a defendant-witness’s credibility will help reduce this burden, bring the rule in line with its original intent, and promote a fairer adversarial process.

Rule 609(a)(1)(B) renders admissible a defendant-witness’s prior felony conviction that did not involve dishonest acts or false statements if the probative value of the conviction, for purposes of attacking the defendant-witness’s character for truthfulness, outweighs its prejudicial effect.[3] This standard is higher than the generally applicable Rule 403 test (that relevant evidence may be excluded if its probative value is substantially outweighed by its risk of unfair prejudice),[4] and marked a departure from the prevailing practice in federal courts at the time of the rule’s enactment, which generally permitted impeachment by prior felony convictions.[5] This change reflected Congress’s intent to ameliorate the “unfair prejudice” of such impeachment and the corresponding “deterrent effect” “upon an accused who might wish to testify.”[6] As the Supreme Court recognized, “the danger is obvious that the jury, despite instructions, will give more heed to the past convictions as evidence that the accused is the kind of man who would commit the crime on charge, or even that he ought to be put away without too much concern with present guilt, than they will to its legitimate bearing on credibility.”[7] This substantial risk dissuaded many defendants, including innocent defendants, from testifying on their own behalf.[8]

Despite the motivations behind Rule 609, however, in practice courts have frequently applied Rule 609(a)(1)(B) in such a way as to permit the admission of highly prejudicial prior criminal convictions with little to no relevance to the defendant’s credibility. Professor Jeffrey Bellin has catalogued how courts overlaid a judicial gloss on Rule 609(a)(1) that transformed what was intended to be a restriction on the admission of defendants’ prior convictions into a justification for their commonplace admission.[9] In a memorandum to the Advisory Committee, Professor Daniel J. Capra similarly described how, although the “balancing test of Rule 609(a)(1)(B) was intended to be protective” of criminal defendants, “the sad fact is that many courts routinely admit” convictions unrelated to dishonest acts or false statements, “even if they are for inflammatory acts, or are for crimes identical to that charged, and even admitting multiple convictions.”[10] Professor Capra enumerated numerous examples of courts admitting or upholding the admission of highly prejudicial prior convictions despite or in apparent violation of the balancing test under Rule 609(a)(1)(B), and noted there are “many” more such cases.[11]

This flawed application of Rule 609(a)(1) is doubly prejudicial because trial court decisions that incorrectly permit the admission of prior convictions often avoid appellate review. The Supreme Court has held that to preserve the issue for appeal, a defendant must testify and be impeached with the prior conviction on cross-examination (and be convicted).[12]

As a result, defendants faced with a court ruling that their past criminal convictions will be admitted at trial, but only if they testify, frequently choose not to take the stand in their own defense, wagering that their silence is less prejudicial than the jury hearing their version of events but also about their prior convictions. Such reasoning is well-founded. A study by Professors Theodore Eisenberg and Valerie P. Hans found that jurors are more likely to convict a defendant with a prior criminal conviction than in an identical case where there is no information about a prior conviction.[13] This problem is exacerbated for people of color, who disproportionately have prior criminal records and, in many federal jurisdictions, are likely to be tried before juries that are disproportionately white.[14] Knowing that if they testify, the jury is going to hear about their prior convictions, which will reinforce preexisting biases about criminal propensity, many defendants choose not to exercise their right to testify in their own defense (which by itself may serve to reinforce jurors’ negative biases).[15]

The results can be tragic. A study by Professor John Blume of convicted defendants who were later exonerated by DNA testing showed that 39% of these individuals did not testify at their own trial and that 91% of this group had a prior criminal conviction.[16] While it is impossible to say that in each of these cases, had the defendant testified, a wrongful conviction could have been avoided, reducing the burden on the introduction of incredibly relevant and powerfully probative evidence—the defendant’s own testimony—is a salutary goal and one which motivated the adoption of Rule 609 in the first place.

The Federal Courts Committee thus supports the proposed amendments to Rule 609(a)(1)(B), which would clarify that the probative value of a prior conviction must substantially outweigh its prejudicial effect before it is admissible. This amendment will hopefully restore the intended balance that Congress meant to strike when Rule 609 was enacted.

The Federal Courts Committee believes that further reform, if not full abrogation, of Rule 609 is warranted, for many of the reasons set forth by the commenters cited herein, and urges the Advisory Committee to consider such changes in the future.

Respectfully submitted,

Jonathan New, Chair
Federal Courts Committee

Drafting Subcommittee
Sarah Dowd, Chair
Jean-David Barnea
James C. Francis IV
Alexander C. Haden
Neil P. Kelly
Robert O’Hare
Jacob Popeck
Jonathan Sabin
Barbara Seniawski
Danielle Tarantolo
Julie van Westendorp