In-depth practical guides

Verify an AI summary before using it

Link important claims to source passages and check numbers, versions, omissions and uncertainty.

Updated :

Summary and sources side by side. Illustrative scene.

AI-generated illustration

Does the text summarise documents or add conclusions?

Fluent summaries may invent references, misattribute facts or add unsupported links. The NIST generative AI profile addresses confabulation risk. This method checks reading and provenance, not truth by writing style.

  1. Split claims

    Identify statements about facts, numbers, causation or identity. Separate document statements from inference; prioritise claims affecting the decision.

  2. Open cited items

    Check title, author, identifier, date and version. Find the exact supporting passage; an existing URL alone is not evidence.

  3. Recalculate and compare

    Check units, denominators, periods and change signs. Review negations, qualifications, excluded populations and association versus causation.

  4. Expose differences

    Mark claims supported, contradicted or unestablished in the corpus. Revise or seek another item; retain original output and correction reasons.

Reading workshop

Review four claims, not the fluency of the summary

The DEMO report and summary are invented. Classifications state what the supplied passages support, rather than universal truth.

Fictional data and records for learning: none of these results comes from a live service query.

An existing source would not suffice: find the passage

The dossier contains March and June versions. Every row below uses the June version; June findings must not silently be attributed to March. The excerpts are fictional.

If the table exceeds the screen, scroll horizontally; arrow keys work when the region is focused.

An existing source would not suffice: find the passage
Summary claimJune DEMO passageClass in this reviewReview action
The pilot involves volunteers“Test conducted only with volunteers.”Supported in this versionKeep the volunteer population and pilot scope.
The results received an external audit“No external audit was performed.”Contradicted by the passageRestore the negation; remove invented validation.
Costs fell by 25%“Costs are not collected in this report.”Not established in this corpusRemove the figure or request a cost record.
The tool caused the improvement“Improvement observed after the change; no control group.”Added causal inferenceDescribe the observation without claiming a demonstrated cause.

Which rewrite best respects this small corpus?

Open each proposition to compare its diagnosis. More checks may still be needed.

The pilot proves a general gain confirmed by audit

No. The population is limited and the passage denies an audit. This combines overgeneralization and contradiction.

Among volunteers in the June pilot, improvement is observed; external audit absent and causality not established

Yes. This retains population, version and qualifications. Keep the undocumented cost figure out of the summary.

Delete every sentence because one is false

This review assesses individual claims. One contradiction does not automatically refute separate supported propositions.

Fictional review record

Source: DEMO report, June version
Supported: pilot limited to volunteers
Contradicted: claim of an external audit
Not established: 25% cost reduction
Inference: causality attributed to the tool
Next: retain passages, original answer and corrected text

The March version remains a separate record. “Not established” does not mean “refuted”. These passages illustrate human reading, without automatic verdicts.

Documentation of the rules, separate from fictional examples

Compare other situations and exercises

Real reference, absent claim

In a fictional case, a page lacks the reported figure. Mark the claim unestablished and locate the exact table; domain identity does not validate it.

Lost qualification

An illustrative summary changes “in this sample” to “in all cases”. Restore population and limits even if the sentence becomes less categorical.

Correct direction, incorrect unit

Fictional example: a document gives a share rising from 20% to 30%, but the summary calls it a 10% rise. Recalculate: it is 10 percentage points, or a 50% increase relative to the starting level. Choose the expression matching the question and retain both source values so the correction can be checked.

Two versions produce an impossible claim

In an illustrative dossier, a January version states a target and a June version reports a different achievement. The summary attributes the achievement to January by combining passages. Match each fact to its version and date; write two dated statements instead of a sentence supported by neither record.

Worked exercise

An observed difference becomes a causal effect

Fictional example: the values and documents teach the method and do not describe a real organisation.

Groups were selected differently. The summary attributes their difference to a causal effect of the tool.

If the table exceeds the screen, scroll horizontally; arrow keys work when the region is focused.

Starting information
GroupSuccessesCount
Using the tool810
Other group610

Your checks

  1. Recalculate rates and their difference.
  2. Read group selection and comparability.
  3. Separate observation from causal interpretation.
Read the answer

Answer and decision

Observed rates are 80% and 60%, a 20-point gap. The supplied file does not establish that the tool causes it: selection and other differences are uncontrolled. Correct the claim to a descriptive comparison with that limitation.

If the case changes

If a causal-inference method is documented, examine its assumptions instead of automatically repeating its conclusion.

Record to retain

Keep counts, selection, protocol and corrected claim with its evidence level.

Record to retain

  • Original output and production context
  • Claims and source passages
  • Recalculated values and versions
  • Differences, correction, owner and date
  • Variation units, each passage’s version and history of corrected wording.

Official references

Related source profiles

Related protocols

Checks before reaching a conclusion

  • Does each important fact have a passage?
  • Were numbers recalculated?
  • Are population limits retained?
  • Is inference separate from source content?

A claim, passage, version and decision matrix with corrected text.

Frequently asked questions

Do several cited links automatically make an answer reliable?

No. Each link must support its associated passage. Repeated links to one source are not independent confirmations.

What if the source is inaccessible?

Retain URL, date and access state. Mark the claim unchecked in this review; seek an accessible version without treating access failure as refutation.

How can corrections preserve the verification trail?

Keep the original answer, problematic passage, examined record and revised wording. An unsupported claim can leave the summary while remaining in the control log. Record the reason without turning absence of support into a refutation.

Should one check every phrase of the summary?

Priorize the statements that influence the decision, the figures and the sensitive proposals. A major error can justify a broader control of the synthesis and its production method.

TOOLS / REGISTERS

Information to record

Reviewing a small corpus certifies neither all model behaviour nor facts beyond the documents. Do not send confidential records to a third-party tool without an appropriate access framework.

  • Access date and time zone
  • Exact URL
  • Claim to verify
  • Interface / mode / account
  • Version / revision
  • Observed passage / value
  • Limit / uncertainty
  • Decision / next action
  • Review owner
  • Next review

Apply the method to specific cases

Proposed exercise: this protocol is not a performed service test.

Record observations

DOCUMENTED REFERENCE

Citing a passage: separate selection from source state

A text selector answers “which passage?”; a state answers “which representation?”. Keeping both helps an evidence record or AI-summary check avoid attributing a quotation to the wrong version.

Citing a passage: separate selection from source state
ReferenceValue or formWhat it identifies
selectorSpecificResource.selectorA selector describes a source segment. On its own, it does not freeze a changing page.
state / cachedTimeState · cachedTimeState can describe the appropriate time and a copy of the representation. cached links to that copy; it certifies neither integrity nor availability.
sourceDate · W3C example 312015-07-20T13:30:00ZW3C requires sourceDate to be an xsd:dateTime in UTC using Z. Its example 31 uses 2015-07-20T13:30:00Z; this belongs to that example, not an observation on this site.
HttpRequestStateAccept · Accept-LanguageHttpRequestState preserves request headers needed for the representation. Accept or Accept-Language is distinct from a capture date.
Source, state and passage are distinct referencesSource identifies the resource, state describes the representation to recover, and selector identifies the passage. This diagram is not a timeline.sourcestateselectorSpecificResource
Source identifies the resource, state describes the representation to recover, and selector identifies the passage. This diagram is not a timeline.

Implication for the record

Neither selector nor state proves a claim. Connect the URL, passage, preserved version, extraction and integrity check actually performed. Keep a translation distinct from the original-language passage.

Are a URL and timestamp enough to recover identical text?

No. The server may no longer supply that state, and representation can depend on requested language or format. Check the copy actually obtained and record text transformations before recomputing offsets.

Primary documentation

Documentation consulted on .

DEEPER READING / PRACTICE

Compare a summary with its source documents

A summary seems clear but changes a reserve or gathers incompatible numbers. The verification must start with important proposals and find out the passages that support them.

Updated on .

Priority checks

Cut the summary into affirmations, open the sources and control scope, date, unit and level of certainty. Release the omissions that change the meaning of the original document.

Interpretation pitfalls

A reference present at the end of the text may support only a part of the statement. The fluidity of the writing does not give any indication of the accuracy of the approaches.

Working record to retain

The verification matrix combines affirmation, source passage, concordance and correction. Non-established items shall be reformulated or explicitly suspended.