AI Hallucination in Legal Cases: What Happened and How to Avoid It

AI hallucination in legal cases happens when a generative AI tool produces citations, quotations, or holdings that look real but do not exist or are inaccurate. It became national news when lawyers were sanctioned for filing a brief full of fake cases invented by a chatbot, and research has since found that even paid legal AI tools hallucinate a meaningful share of the time. The defense is straightforward: verify every citation at its source before you use it.

AI is genuinely useful for legal research, but it has a failure mode that is uniquely dangerous in law, where a single fabricated citation can sink a filing and a reputation. Understanding why hallucinations happen, and what the record actually shows, is the first step to using these tools safely.

What an AI legal hallucination is

A large language model generates text by predicting plausible sequences of words, not by looking up facts in a verified database. That design makes it fluent and fast, but it also means the model can produce a citation that fits the pattern of a real one, right down to a believable volume, reporter, page, and year, without any actual case behind it. The output looks authoritative precisely because the model is good at imitating the form of legal writing.

In law this is especially hazardous. A fabricated case can carry a real-sounding name, a plausible citation, and even an invented holding and quotation. Nothing on the surface warns you it is fake. You only find out by trying to pull the case up and verify it.

Mata v. Avianca: the case that put fake citations on the map

The most cited real-world example is Mata v. Avianca, a 2023 matter in the Southern District of New York. Lawyers submitted a brief that relied on several cases that turned out not to exist. The fake cases had been produced by a general-purpose AI chatbot, which the attorneys had used for research and had not independently verified. When opposing counsel and the court could not locate the decisions, the problem unraveled.

The court sanctioned the attorneys involved. The episode became a national warning about using generative AI in legal work without verification. The lesson lawyers drew was not "never use AI," but "never file anything you have not confirmed exists and says what you claim." Mata v. Avianca is a real case; the fabricated cases cited in it were not.

What the Stanford research found

It would be comforting to think fabrications only come from generic chatbots and that purpose-built legal tools are immune. Research from Stanford's RegLab and related work complicated that assumption. Studies evaluating legal AI tools found hallucination rates that were far from trivial, on the order of roughly 17 to 33 percent in the tools tested, even for products marketed specifically for legal research.

The takeaway is nuanced. Purpose-built and retrieval-grounded tools tend to hallucinate less than a raw chatbot, which is real progress. But "less" is not "none." A tool that is wrong a meaningful share of the time still requires human verification of every citation. Anyone who tells you a legal AI product is completely free of hallucination is overselling. We do not make that claim about Cases either.

Why grounding and source links reduce, but do not eliminate, the risk

The most effective mitigation is grounding: designing the tool so its answers are tied to actual retrieved documents rather than free-form generation, and linking each claim to the real source. When a result comes with a citation that opens the actual opinion, a fabrication is obvious, because the link either shows the real case or it does not resolve.

This is the core idea behind checkable citations in Cases: every result points to a source you can open and confirm. Grounding shifts the odds heavily in your favor and makes verification fast. It does not, however, remove your obligation to verify. A linked source can still be misread, mischaracterized, or taken out of context, so you still read the opinion yourself.

How to protect yourself from AI hallucinations

Whatever tool you use, a few habits keep you safe:

1. Verify that the case exists. Pull it up in an authoritative reporter or database. If a citation will not resolve, treat it as fake until proven otherwise. Our guide on how to find case law explains where to look.
2. Read the actual holding. Confirm the court really held what the tool says. Do not rely on an AI summary alone for the substance of a rule.
3. Check every quotation against the opinion. A quote that does not appear on the cited page is a red flag.
4. Confirm it is still good law. Run a good-law check, as described in how to Shepardize a case. A real case that has been overruled is its own kind of trap.
5. Prefer tools that link to sources. A tool that hands you a bare citation with no link is asking you to trust it blindly. A tool that links to the real opinion invites verification.
6. Never file what you have not personally confirmed. The buck stops with the human, as Mata v. Avianca made painfully clear.

Why lawyers are especially exposed

Every profession faces AI errors, but law is unusually unforgiving for a few reasons. First, filings become part of the public record and are scrutinized by opposing counsel whose job is to find the weak spot. Second, courts impose duties of candor, so a fabricated citation is not just an error but a potential ethical violation. Third, the form of a legal citation is easy to imitate and hard to spot-check by eye, so a fake citation blends in until someone tries to pull the case. Finally, a single bad citation can taint an entire brief, because it signals that the rest of the work may not have been verified either. These pressures are exactly why the human-verification step is non-negotiable in legal work, even more than in casual research.

Beyond fabrication: subtler AI errors

Outright invented cases are the dramatic failure, but subtler errors are just as dangerous and easier to miss:

• Real case, wrong holding. The tool cites a genuine case but describes a holding it does not contain.
• Real case, overruled. The citation exists and once stated the rule, but the case has since been overruled or superseded.
• Real case, misquoted. A quotation is attributed to the case but does not appear on the cited page, or is stitched together from different passages.
• Right rule, wrong jurisdiction. The case is real and correctly summarized but comes from a court that does not bind yours, so it is persuasive at best.

Each of these passes a quick glance and fails a careful read. That is why verifying means opening the opinion and confirming the holding, the quote, the jurisdiction, and the good-law status, not just confirming the case exists.

Using AI research responsibly

None of this means avoiding AI. Used well, AI-assisted and plain-English research finds relevant precedents faster than boolean search and lowers the barrier for solos, paralegals, students, and self-represented litigants. The point is to treat AI as a research assistant that surfaces candidates, not as an oracle that delivers final answers. The human reads, verifies, and decides.

That is exactly how Cases is designed to be used: ask a question in plain English, get precedents with summaries and holdings, and follow each checkable citation to confirm it. The tool makes verification trivial; it does not do the verifying for you.

What courts and bar authorities now expect

Since the fake-citation episodes made news, courts and professional authorities have made their expectations explicit. Some judges now require lawyers to certify that they have verified any AI-assisted content, and bar guidance increasingly stresses that using AI does not lessen a lawyer's duty of competence and candor. The through-line is consistency with duties that already existed: a lawyer has always been responsible for the accuracy of what they file, and AI does not change that. If anything, these developments formalize the point of this article. The tool can help you research, but you sign the filing, and you are accountable for every citation in it. Treating AI output as a draft to verify, rather than a finished product to submit, keeps you on the right side of that duty.

The honest boundary

We will say it plainly and we will not dress it up: no AI legal research tool is completely free of hallucination, and Cases does not claim to be. Every tool, including Cases, is for informational research only, not legal advice. Always verify the citation at its source, read the holding yourself, and confirm the case is still good law before you rely on it. The safest lawyer is the one who assumes the machine can be wrong and checks.

Key takeaways

• AI hallucinations are fabricated or inaccurate citations, quotations, or holdings that look real.
• Mata v. Avianca (2023) led to sanctions for filing fake AI-generated cases.
• Studies found legal AI tools still hallucinate on the order of 17 to 33 percent, so "less" is not "none."
• Grounding and source links reduce the risk and make verification fast, but do not remove your duty to verify.
• Verify every citation exists, says what you claim, and is still good law before relying on it.