Cross-Border Tokenization: Bridging Multiple Jurisdictions
Unlock Premium to read the full edition. Subscribe
This is a summary of one research paper. For our analysis, read the newsletter →
Papers › AI cites the right authority but decides without it
AI cites the right authority but decides without it
Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness
Saisab Sadhu, Shreeyans Arora, Pratinav Seth · submitted 8 Oct 2026
Practical takeaway
- Across seven open-weight models, the AI named the correct statute or precedent in 66.7–100% of answers, yet swapping that authority for an unrelated one often did not change the verdict.
- A correct citation is therefore weak evidence that the conclusion actually rests on it.
- Hidden adversarial instructions in the case facts were followed 73–96% of the time, so untrusted documents fed to a model are a real attack surface.
- Do not use AI-generated legal reasoning as a compliance or audit artefact without independent human checking of the authority-to-outcome link.
Original abstract
Large language models increasingly justify legal decisions by naming the statute or precedent behind a verdict, treated as evidence that the decision follows from it. We test this directly: holding case facts fixed, we substitute the named legal authority for an unrelated one and decode a model's evolving verdict from its hidden states. Across seven open-weight models (8B-70B) and four benchmarks spanning judicial and contractual reasoning, when explicitly required to justify a verdict by naming the governing authority, models name the correct one in 66.7%-100% of generations, while the verdict changing when the authority changes is far less consistent: 0.0%-21.7% on CaseHOLD, 30.0%-76.7% on ECHR and SCOTUS, and 43.3%-50.0% on ContractNLI. Neither scale nor a purpose-built legal-reasoning model (a best-effort LoRA reproduction; Section 6) closes this gap. A red-teaming evaluation on five core models finds compliance with an adversarial instruction hidden in the case facts (73.3%-96.4%) exceeds verdict-swap sensitivity by a wide margin, holding without exception across model rankings. Naming a legal authority is thus a poor proxy for a verdict's dependence on it, while the same verdict remains separately vulnerable to adversarial manipulation. Both findings replicate across checks ruling out prompt-wording noise and confounded sampling, and bear directly on the use of generated legal explanations as compliance or audit artefacts.
Paper details
- Authors
- Saisab Sadhu, Shreeyans Arora, Pratinav Seth
- Source and ID
- arXiv 2610.12361
- Primary category
- cs.AI · AI in law
- Last verified
- 18 Oct 2026
From the newsletter
More in the series →Related editions of Tokenization Architect, in English.
Trade Documentation in Tokenization: Why Paperwork Is the Asset
Unlock Premium to read the full edition. Subscribe
All papers
- PARCEL: phantom citations: When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection
- The RWA liquidity myth: When Token Transfers Do Not Mean Liquidity in Tokenized Real-World Assets