Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness

A counterfactual audit evaluating legal chain-of-thought reasoning across seven open-weight large language models (8B to 70B) reveals that models frequently cite legal authorities without utilizing them to determine verdicts. While models cite the correct legal authority in 66.7% to 100% of generations across judicial and contractual benchmarks, altering the cited precedent or statute causes the verdict to change only inconsistently, proving that cited authorities do not reliably govern model conclusions.

This matters because relying on AI-generated legal citations creates a false sense of procedural rigor and auditability. The evidence demonstrates that large language models produce superficially valid justifications that do not causally drive their underlying determinations. Consequently, organizations risking automated legal or compliance decisions face significant liability if reasoning cannot be counterfactually verified.

Evaluated seven open-weight models (8B to 70B parameters) across four legal benchmarks covering judicial and contractual domains: CaseHOLD, ECHR, SCOTUS, and ContractNLI.

Forward consideration, not a verified fact.