When AI invents a case
“Hallucination” covers at least six different failures, and they do not look alike. The dangerous ones are not the obviously fake citations. Here is what each one looks like in England & Wales work, and how to catch it.
Every lawyer who has touched a general-purpose AI tool knows it can make up cases. What gets less attention is that “making things up” is not one failure. It is a family of them, and the worst ones do not look like fabrication at all. They look like good drafting.
We wrote a technical report on this (TR-2026-01) because our own product needs a way to score these failures separately. This post is the same material without the apparatus: what each failure looks like when you are staring at model output, and the checks that catch it.
The obvious one
The famous failure is the invented authority. A citation appears, correctly formatted, plausibly dated, sometimes with a pinpoint. It resolves to nothing. The neutral citation does not exist; the party names match no report. This is the one that made the news when US lawyers filed briefs full of them, and it is the one most tools have learned to catch, because it is easy to check. Does the citation resolve to a known record? No. Flag it.
The dangerous one
Worse is the real case cited for a point it does not support. The case exists. It is even in the right area of law. But it stands for something else, or it stands for the opposite of the proposition attached to it. String-matching the citation finds nothing wrong, because the citation is genuine. You only catch this by reading the case against the claim, which is exactly the step the model was supposed to save you.
This failure survives a first skim almost perfectly. The citation looks professional because it is professional. It is also the failure most likely to reach a court, precisely because nobody suspects it.
The sloppy ones
Citation form errors sit between the two. The case is right and supports the point, but the neutral citation is mangled, the party names are wrong, or the pinpoint sends you to the wrong paragraph. It is not dishonest output so much as unreliable output, and it costs verification time every time it happens. If a product produces these often, treat the rest of its citations with suspicion too.
Quote drift is subtler. The quotation in the output does not match the judgment after you allow for typographic tidying. Words dropped, words rearranged, occasionally a word swapped for a near-synonym that changes the holding. A quote that reads naturally is the easiest thing in the world to skim past. Diff it against the source paragraph. That is the only check.
The two that look like nothing
The unsupported proposition is the hardest failure to see, because every component is correct. The case is real, the citation is clean, the quote is faithful. But the passage quoted does not support the proposition it is attached to. The model found a true thing and asserted it proves something it does not. No string check catches this. Someone has to read the passage and ask whether it actually carries the claim.
And the quietest failure is the retrieval miss. The right source existed in the corpus and the system never found it, so the output is built on a weaker authority or none at all. This does not show up as an error in the text. It shows up as an absence, and absences are invisible until you know what should have been there.
Why a single score hides all of this
If a vendor tells you their citation handling is 95% accurate, ask what is in the other 5%. A system that invents one authority in twenty is a different product from one that drifts a quote in twenty, and a headline number cannot tell them apart. This is why our benchmark suites score each failure mode separately, and why none of them publish until the suite is frozen. An average of unlike errors is a marketing figure.
A sixty-second check that works
Whatever tool produced the output, three questions catch most of it. Does each citation resolve to a real judgment? Does each quote match the source paragraph word for word? Does the passage actually say what the sentence claims it says? The first two are mechanical. The third takes a minute of reading and catches the failures that matter most.
What we are doing about it
Verify is the part of the workspace built around this taxonomy. Run a matter document through it and it checks each citation against the stored corpus: does the authority resolve, is it the right authority for the point, does the quote match the stored paragraph after normalisation. The unsupported-proposition check is the hard one and we treat it accordingly. Verify flags it for review; it does not pretend to settle it.
The measurement side is the benchmark programme. Each failure mode above is scored separately in the citation resolution suite, which is internal at v0.3 and stays internal until it freezes. When a suite freezes we publish methodology, manifests, and scores together. A number without a replayable suite behind it is just marketing.
Underneath both sits the stored corpus. Verify can check quotes word for word because the paragraphs it checks against are stored pages, not links that might rot. Reproducible retrieval is what turns “the model said so” into something a firm can audit.
None of this removes the lawyer from the loop, and it is not supposed to. The aim is that the mechanical checks stop costing mechanical time, so the reading that matters gets done properly. If a tool claims it has fully automated citation trust, that claim belongs in the same bin as the invented authorities.