In one sentence
AI can invent legal citations that look convincingly real — you can never rely on the reference alone, only on a process that checks every citation against a genuine source before the pleading leaves the office.
The facts in 2026
The use of AI in law firms has long been the norm: according to Wolters Kluwer's Future Ready Lawyer 2026, 63.6% of German law firms actively use AI — two years ago the figure was below 30%. As adoption grows, so does the risk. In a ruling of 20 November 2025 (17 WF 144/25), the Kammergericht Berlin (Berlin's Higher Regional Court) reprimanded a lawyer because a pleading contained fabricated citations, and it set out the duty of professional scrutiny (report on beck-aktuell). This is no isolated case: internationally, around 1,600 AI hallucination cases have been documented across 35 countries, and courts — particularly in the United States — have repeatedly imposed sanctions. You can read our assessment of the Berlin case in our news article on the Kammergericht Berlin.
Why language models hallucinate
A language model does not query a database — it predicts the next statistically plausible word. A fabricated case number therefore looks just like a real one: correct court, correct format, plausible date. That is precisely what makes AI hallucinations in judgments so dangerous. The obvious reflex — telling the model via a prompt to "please cite only real judgments" — does not help reliably: at the moment of generation the model has no arbiter of truth and cannot tell its own inventions apart from genuine recollections. We explore why pure chat tools reach their limits here under Why not ChatGPT.
The three-stage safeguard
AI becomes reliable in a pleading only through a process, not through better prompts. A three-stage safeguard has proven itself for avoiding hallucinations in the firm:
- A source requirement in the tool: the tool outputs no citation without a documented source — a claim without evidence is flagged, not passed through.
- Automatic citation verification: every provision and every decision is checked against genuine registers and a real body of citations, rather than taken from the model's memory.
- A final human check: before dispatch, a person reviews the work — professional responsibility remains undivided.
The EU transparency obligations (Art. 50 EU AI Act) have applied since 2 August 2026, yet a label does not make a citation true: the burden of verification stays with the firm.
How SmartLegalPro secures citations
SmartLegalPro does not replace lawyers — the tools are designed to carry the three-stage safeguard technically:
- Source completion (Verify & Expand): checks the citations in a draft against documented sources and adds references instead of inventing them (more on this).
- Rechtsbibliothek with a genuine graph of provisions and decisions: a knowledge graph against which every citation can be reconciled with real evidence (to the Rechtsbibliothek).
- Deadline double-check: an independent second look at calculated deadlines — figures and dates are just as prone to hallucination as citations (details).
Added to this are EU hosting, quota-based rather than euro-based billing in the Anwaltsportal, and the principle that client data is never used for training.
Five checks before sending
- Look up every citation: verify the case number and the section in an official source such as gesetze-im-internet.de — do not merely check for plausibility.
- Read the core proposition against the source: does the cited decision really support what you are citing it for?
- Calculate deadlines twice: have the dates recalculated independently and reconcile them with the deadline calendar.
- Flag passages without a source: anything without evidence counts as unverified until a source is available.
- Document the sign-off: who checked, and when? A brief note on the final review of the pleading protects both the client's matter and the firm.
AI speeds up legal work considerably — but responsibility for every citation remains human. A clean process of source requirement, citation verification and a final check is the difference between an efficiency gain and a reprimand.
