Evaluation field guide

How to detect AI hallucinations

A source-linked detection stack that combines retrieval evidence, atomic claims, entailment checks, uncertainty, evaluation sets and human review.

No single score proves an answer is factual. Detection works as a layered decision path that can cite, verify, abstain and escalate.

For teams evaluating factual answers, long-form generation or retrieval-augmented systems.

Open the full research desk

7defensive checkpoints

8authoritative sources

24 Aug 2026Evidence checked

21 Aug 2026Content reviewed

Hallucination

Hallucination detection method stack

Combine evidence, uncertainty and accountable review instead of trusting one score.1 2 3 4 5

Failure path

A fluent answer can contain unsupported, contradicted, outdated or internally inconsistent claims that users may accept without checking.1 2 3 4 5

Prevent

Limit generation to supported tasks, retrieve approved evidence, require citations, permit abstention and narrow output claims.1 2 3 4 5

Detect

Use a layered stack of retrieval support, atomic claims, entailment, sample consistency, semantic uncertainty and human review.1 2 3 4 5

Validate

Evaluate each layer and the combined decision policy on representative, time-split and adversarial examples with human labels.1 2 3 4 5

Known limits

No method covers every failure class; detectors can share data, model and retrieval errors with the generator.1 2 3 4 5

Operational sequence

Decompose the answer, retrieve evidence, score support and uncertainty, block high-risk claims, then send unresolved high-impact items to review.1 2 3 4 5

Evidence for this checkpoint

Hallucination

Retrieval grounding and citation alignment

Make every material claim traceable to the passage that supports it.1 2 3

Failure path

A model can ignore retrieved evidence, cite an irrelevant passage or combine sources into a claim none of them supports.1 2 3

Prevent

Retrieve from approved current corpora, preserve passage identifiers, constrain answers to evidence and allow a no-support outcome.1 2 3

Detect

Score context relevance, claim support and citation coverage separately; flag citations that do not entail the attached claim.1 2 3

Validate

Use answerable and unanswerable queries, stale evidence and distractor passages, then compare automated support labels with reviewers.1 2 3

Known limits

Grounding cannot rescue a missing, incorrect or poisoned corpus and does not prove the source itself is true.1 2 3

Operational sequence

Classify the question, retrieve current approved passages, answer only supported claims, attach exact citations and abstain when coverage is insufficient.1 2 3

Evidence for this checkpoint

Hallucination

Atomic claim decomposition

Break long prose into claims that can each be checked.1 2 3

Failure path

Document-level scores hide mixed outputs where several accurate statements surround one unsupported high-impact claim.1 2 3

Prevent

Ask for bounded claims and structured fields, separate facts from interpretation and require evidence identifiers for factual fields.1 2 3

Detect

Decompose sentences into independently verifiable units and label each supported, contradicted, unsupported or undecidable.1 2 3

Validate

Measure decomposition completeness and claim labels against dual human annotation, especially for numbers, dates and negation.1 2 3

Known limits

Decomposition can remove context, split claims incorrectly or create units whose truth depends on unstated assumptions.1 2 3

Operational sequence

Extract claims, preserve sentence context, retrieve evidence per claim, adjudicate labels and block the document if critical support is missing.1 2 3

Evidence for this checkpoint

Hallucination

Entailment-based evidence checking

Distinguish support from contradiction and missing evidence.1 2 3

Failure path

Keyword overlap can make a citation appear relevant even when it contradicts or fails to establish the generated claim.1 2 3

Prevent

Keep evidence passages short and attributable, normalize dates and units and require undecidable as an explicit outcome.1 2 3

Detect

Run natural-language inference for each claim-passage pair and aggregate conservatively across independent sources.1 2 3

Validate

Evaluate domain, language, numerical, temporal and negation slices; calibrate thresholds against human adjudication.1 2 3

Known limits

NLI models inherit training-domain limits and may treat lexical similarity as evidence.1 2 3

Operational sequence

Pair each atomic claim with its cited passage, classify support, keep undecidable separate and escalate contradictions immediately.1 2 3

Evidence for this checkpoint

Hallucination

Self-consistency and semantic uncertainty

Use answer variation to find confabulation risk, not to vote truth into existence.1 2 3

Failure path

When the model lacks stable knowledge, repeated generations can express incompatible meanings while each sounds confident.1 2 3

Prevent

Allow abstention, retrieve evidence before generation and keep high-uncertainty tasks outside automated decision paths.1 2 3

Detect

Sample multiple answers, cluster semantic equivalents and measure disagreement with SelfCheckGPT-style or semantic-entropy signals.1 2 3

Validate

Calibrate on in-domain factual labels and test systematic falsehoods separately because consistent error can look certain.1 2 3

Known limits

Sampling is costly, semantic clustering can be wrong and low uncertainty does not establish factuality.1 2 3

Operational sequence

Generate controlled samples, normalize meanings, compute disagreement, combine with evidence support and refuse or review above threshold.1 2 3

Evidence for this checkpoint

Hallucination

Hallucination evaluation sets and drift

Measure the failures your product sees, then keep the test set moving with reality.1 2 3 4

Failure path

Generic benchmarks can hide domain, language, time and workflow failures that dominate a real deployment.1 2 3 4

Prevent

Create representative test strata, preserve fresh holdouts, document label policy and prevent evaluation examples from entering training.1 2 3 4

Detect

Track factual precision, unsupported-claim rate, abstention, citation coverage and severity by domain, language and model version.1 2 3 4

Validate

Use blinded dual review, adjudicate disagreement, report confidence intervals and rerun time-split holdouts after releases.1 2 3 4

Known limits

Benchmarks become stale, can be contaminated and rarely predict every live failure mode.1 2 3 4

Operational sequence

Sample production-shaped prompts without personal data, label with evidence, freeze a release gate, monitor drift and refresh only with versioned review.1 2 3 4

Evidence for this checkpoint

Hallucination

Human review and calibrated abstention

Define when the system stops and who owns the unresolved claim.1 2 3 4

Failure path

Automated checks can pass plausible errors, while unstructured review can become inconsistent, slow or overly trusting of fluent text.1 2 3 4

Prevent

Set risk tiers, evidence requirements, reviewer qualifications, service levels and a safe abstention response before launch.1 2 3 4

Detect

Route cases using claim severity, missing support, contradiction, uncertainty and detector disagreement rather than model confidence alone.1 2 3 4

Validate

Measure reviewer agreement, correction rate, queue delay, anchoring and outcomes after release; audit a random sample of passes.1 2 3 4

Known limits

Human review does not guarantee truth and can fail under overload, weak evidence or ambiguous accountability.1 2 3 4

Operational sequence

Apply automated gates, abstain on critical uncertainty, present claims beside evidence, record reviewer decisions and feed confirmed errors into regression tests.1 2 3 4

Evidence for this checkpoint

Continue the research

Search all 21 defensive dossiers, compare hallucination-control methods and inspect every source record.

Open the full research desk