Structured Policy Analysis

RAG and Hallucination Insights

Why retrieval-augmented generation reduces but does not fix LLM hallucination. AI research grounded in evidence, structured by causal mechanisms. Independent verification required.

0claims analyzed

0sources cited

0causal mechanisms

Key Findings

Retrieval-augmented generation can substantially reduce hallucination, but it does not eliminate it. Even a purpose-built legal AI that retrieves a correct source still produces unsupported answers in roughly 17 to 33 percent of high-stakes queries. The naive rule that better retrieval means fewer hallucinations is not reliable: in controlled tests, the retriever's top-ranked but off-answer passages can lower accuracy, while random filler text can raise it. Residual failures split into two kinds. The right source may never reach the model, or the right source reaches the model and the answer is still unsupported.

Findings vary by model, domain, retrieval pipeline, and how hallucination is defined and measured. Numbers from one benchmark or task do not transfer cleanly to another, and detection tools themselves are imperfect.

Grounding can fail even with the right source in hand

Studies separate retrieval failure (the right passage never reaches the model) from faithfulness failure (the right passage is present but the answer is unsupported). Both occur, and fixing one does not fix the other.

More context can mean more confident errors

One analysis found a model jumped from wrong on about 10 percent of questions with no context to wrong on about 66 percent with insufficient context. Added context can raise confidence and lower the tendency to abstain.

Random noise sometimes beats the top-ranked passage

In controlled retrieval experiments, adding random documents raised accuracy by up to 35 percent, while high-scoring but off-answer documents hurt. This complicates the assumption that better ranking always reduces hallucination.

Citations can be decorative, not load-bearing

One study found up to 57 percent of model citations were post-rationalized: the cited document did not actually drive the answer. Correct-looking attribution is not the same as faithful grounding.

Hallucination detectors are themselves unreliable

On a benchmark of hard cases, even the best automated hallucination detectors scored near 50 percent accuracy, and expert annotators disagreed on gray-area cases. Measuring faithfulness is itself an open problem.

Zero-hallucination claims are conditional

One method reported 100 percent hallucination elimination on a benchmark, but only for inputs that contained factually correct passages. The claim does not cover retrieval failure or out-of-corpus questions.

Research Findings

Sources

What this means in practice

Work that relies on document-grounded AI often involves manually pulling the right source files, checking whether an answer is actually supported by them, and logging which passage backed each conclusion. These review steps are typically handled with systems that automate the repetitive parts.

Ingest source documents and structured data into one place
Automate retrieval, grounding checks, and tracking of which passage supports each output
Generate clear, repeatable outputs and citations for review and reporting

See example systems