How to Reduce LLM Hallucination

An LLM hallucinates when it states something false with the same fluent confidence it uses for something true. The output reads well, the grammar is clean, the tone is sure, and the fact is wrong. For a chatbot that is an annoyance. For a system that answers customer questions, summarizes contracts, or feeds decisions downstream, it is the failure mode that decides whether you can trust the system at all.

The useful shift is to stop treating hallucination as a defect to wait out and start treating it as a property to engineer around. The next model release will not quietly remove it. It has roots in how models are trained and graded, which means the reliable fixes live in the system you build around the model: ground it in real sources, constrain what it is allowed to claim, verify its output, and measure how often it strays. This post walks through why it happens and what actually moves the number.

Why models hallucinate

The clearest explanation is now peer-reviewed. In work published in *Nature* in April 2026 (an expanded version of OpenAI's 2025 paper "Why Language Models Hallucinate"), researchers argue that hallucination is not a bug bolted on by accident but a predictable outcome of the training objective. A model pretrained to predict the next token comes out of that process as a well-calibrated predictor of text. For facts that follow no learnable pattern and appear rarely in training (an individual's birthday, an obscure citation), there is no signal to compress, so the model can only guess, and it guesses in the same confident voice it uses for everything else.

The sharper point is about why the behavior persists after training. Most benchmarks grade with binary accuracy: a confident wrong answer and an honest "I don't know" both score zero, while a lucky guess scores one. Under that scoreboard, the score-maximizing move is always to guess. Models are optimized to be good test-takers, and good test-takers do not leave questions blank. That reframes the whole problem. Hallucination is less a mystery of the weights than an incentive we trained in, which is also why local changes to the incentive (giving the model permission to abstain, grading it against sources) move the behavior so reliably.

It helps to separate two kinds. A factuality hallucination contradicts the real world. A faithfulness hallucination contradicts the material you actually gave the model. The two have different fixes: retrieval attacks the first by supplying real facts, and tight context plus verification attacks the second by keeping the answer inside the source. Most production techniques are aimed at one or the other.

Ground it in real sources

The single most effective move is to stop asking the model to recall and start asking it to read. Retrieval-augmented generation fetches relevant source material and puts it in front of the model, so the answer comes from documents you control rather than from parametric memory. The fuller mechanics live in what is RAG; the point here is that grounding directly attacks factuality hallucinations by replacing "what does the model remember" with "what does this document say."

Grounding is necessary but not sufficient, and the failure modes are worth naming:

• Retrieval quality is the ceiling. If the wrong chunks come back, the answer is grounded in the wrong thing and sounds just as confident. The system is never better than what retrieval surfaces.
• Restrict the model to the context. Instruct it to answer only from the provided documents and not from general knowledge. Anthropic's own guidance calls this external-knowledge restriction, and it is what keeps the model from quietly topping up gaps with recall.
• Give it a sanctioned way to say "not found." Without an explicit escape hatch, a model asked a question the documents do not answer will fill the gap. Tell it to reply "no relevant information found" when the source is silent, and it stops inventing.
• Cite at the claim level. Have the model attach a supporting quote to each claim so the output is auditable. This is also the hook for the verification step below. One caution, though: even good pipelines have been documented fabricating citations, so a citation is a starting point for a check, not proof on its own.

Constrain what it is allowed to claim

Before reaching for architecture, several prompt-level techniques measurably lower the rate, and they come straight from Anthropic's official guidance on reducing hallucinations rather than folklore:

• Give permission to be uncertain. Explicitly telling the model it is allowed to say "I don't know" drastically reduces false information. This is the incentive fix from the *Nature* paper applied locally: you re-price abstention so it is no longer worse than a guess.
• Ground reasoning in direct quotes. For long documents, have the model first extract word-for-word quotes relevant to the question, then reason only over those quotes. The answer is anchored to text that actually exists.
• Verify and retract. Ask the model to support each claim with a quote and to remove any claim it cannot support, then flag where it was cut. Self-retraction catches a surprising amount before the answer ever leaves the model.
• Lower the temperature for factual work. Less sampling of low-probability tokens means fewer creative detours. This is conventional practice rather than a cited result, but it is cheap and it helps.
• Constrain the output shape. For anything with a fixed set of valid answers, restrict generation to a schema or enum so the model literally cannot emit an out-of-set value.

Verify the output

The techniques above make a single pass more honest. Verification adds a second pass that checks the first, and this is where system design does the heavy lifting.

Chain-of-verification is a clean, published example. The model drafts an answer, then plans a set of verification questions, answers those questions independently so it is not just echoing its own draft, and revises the answer in light of what the checks turned up. In the original study this roughly doubled precision on list-style questions and cut hallucinations in long-form generation. Self-consistency takes a related angle: sample several answers to the same prompt and keep the one they agree on, treating disagreement across runs as a signal that the model is guessing. A third pattern uses a separate model as a critic, LLM-as-a-judge, to score the answer for support against its sources and flag unsupported spans.

The most durable fix, though, is to let the model look things up instead of recalling. A model that can query a database, search a knowledge base, or call a calculator resolves verifiable claims against a live source rather than its weights. Reasoning-and-acting patterns interleave those lookups into the model's chain of thought, so retrieval happens exactly where a fact is needed. Wherever a claim can be checked against ground truth, checking it beats trusting it.

Measure how often it strays

You cannot improve what you do not measure, and hallucination is measurable. The workhorse metric is faithfulness (or groundedness): the fraction of claims in an answer that are actually supported by the retrieved context. The usual method is to have a judge model break the answer into atomic claims and check each one against the source, then score supported over total. Tools like RAGAS and DeepEval compute versions of this, and standing industry benchmarks such as Vectara's summarization-faithfulness leaderboard track it across models. They are a useful reality check, including the counterintuitive finding that heavier reasoning models sometimes do worse on staying faithful to a source even as they do better on math.

Generic leaderboards set expectations; they do not tell you whether your system works. For that you need an eval set built from your own domain and your own documents, including adversarial and unanswerable questions, that measures how often the system abstains correctly, fabricates a citation, or states something false with confidence. That number, specific to your workflow, is the one that catches regressions before your users do.

The honest limit

Every serious source lands in the same place: you can reduce hallucination substantially, but you cannot drive it to zero. The *Nature* result locates part of the cause in the training objective itself, and a related theoretical result argues that for certain classes of rare facts a well-calibrated model must hallucinate at some nonzero rate. Anthropic states it plainly in its own docs: these techniques significantly reduce hallucinations but do not eliminate them, so critical information should always be validated, especially for high-stakes decisions.

There is even a trap in the toolkit. Adding reasoning is not a blanket win: reasoning models sometimes hallucinate more on grounded summarization, and a well-argued chain of thought can make a wrong answer look more trustworthy and harder to catch. No single trick is a solution. The number comes down when the techniques are stacked and, at the steps that matter, a human stays in the loop.

How XY Space thinks about it

XY Space treats hallucination as a system-design problem, which is the only framing that survives contact with production. The reframe from the research is the one we build on: because hallucination is an incentive and grounding problem, you engineer the surrounding workflow to reward answers that are grounded, cited, and honest about uncertainty, rather than waiting for a model that never invents anything. That model is not coming.

In practice that means the same stack every time: ground answers in retrieved sources, instruct the model to work only from those sources and to abstain when they are silent, verify claims against the sources with a second pass or a judge, measure faithfulness continuously on an eval set drawn from the client's own domain, and keep a human in the loop at the consequential steps. That is the shape of every reliable agentic workflow we deploy: structure around the model that turns a confident guesser into a system you can trust with real work. If you want help building one that holds up, talk to us.

Sources

• Why Language Models Hallucinate (arXiv)
• Evaluating large language models for accuracy incentivizes hallucinations (Nature)
• Reduce hallucinations (Anthropic docs)
• Chain-of-Verification Reduces Hallucination (arXiv)
• Calibrated Language Models Must Hallucinate (arXiv)