Sam asks AI to price a team subscription and gets a yearly total that covers a single month.
Sam and the office scenes here are fictional. You can use the same checks on a real draft before sending it to someone else.
A plausible next word can send the answer the wrong way
Sam also asks for a customer help note. The supplied rule allows cancellation before shipment, but the draft says after shipment.
That change could lead the rest of the answer toward return instructions the source never supplied.
Many chat language models generate text by predicting the next token from the prompt and the text already written.
A token can be a word or part of one. Google's language model introduction explains these small pieces of text.
The model learns patterns and facts during training, but a likely continuation does not prove that the statement matches your source.
An early mistake can become part of the context used for later writing. You may get a tidy explanation built around the wrong rule.
Errors can also come from missing information or a bad calculation. One wrong word does not force every later word to be wrong.
Give the reviewer something better than its own draft
Back at the subscription task, Sam asks the same assistant to check its total.
It repeats the number with a reassuring explanation.
You have another answer, but still no calculation you can inspect.
In our made-up price sheet, the service costs $18 per user each month. With eight users, the monthly cost is $144.
If the price and team size stay unchanged, twelve months cost $1,728 before taxes or discounts.
The draft has labeled the $144 monthly total as a yearly total. You can catch that by calculating 18 times 8 times 12 with a calculator.
A 2023 study of reasoning tasks found that self-correction without outside feedback did not reliably improve the tested models and sometimes made answers worse.
That result does not prove every current model will fail. We would ask for the source and calculation before accepting a self-review as evidence.
Write the checks down so you can reuse them
Sam saves the review instructions in a file called REVIEW.md. You can attach the file to a task and explicitly ask the assistant to read it.
If your chat cannot read attachments, paste the rules into your request. Saving a filename on your computer does not make another tool load it.
You can download the Markdown review checklist or copy the text below.
# Review before delivery
- Use the supplied source for each factual claim.
- Recalculate numbers. State units and assumptions.
- Check names, dates and conditions against the source.
- Mark claims without evidence as unverified. Never invent evidence.
- Re-read after edits for changed meaning.
- Report corrections and remaining unknowns.
- Stop after two review rounds. Escalate unresolved claims.Give it the price sheet as well as these rules.
For Sam's task, the useful result is a corrected yearly total with its monthly basis shown.
The two-round limit is an example budget choice, not a research finding. Unresolved claims should reach you rather than become approved through repetition.
We would keep this file short enough to read before each task. You still need to check whether the assistant followed it.
Check the meaning as well as the arithmetic
Suppose a revised help note says cancellation is available at any time. A missing condition has become a promise to the customer.
You need to check the condition in the help note as carefully as the multiplication.
Sam asks the reviewer to compare each policy claim with the supplied rule. It should flag the missing before-shipment condition and restore it.
Ask for a short correction record so you can see what changed and which source supports the change.
Research on Chain-of-Verification explored drafting an answer, asking separate verification questions and then revising it. The study found fewer factual errors on its tested tasks.
Its checks used the model itself, without web search. Separate questions can help, but they do not supply independent outside evidence.
For your note, you already have the rule to compare against. If the document says nothing about returns, leave the return policy unresolved.
Let the writer stop and fetch the missing fact
Picture the assistant as a writer about to label $144 as the yearly cost. A colleague slides over the price sheet.
The writer spots "per user each month", stops the sentence and changes the calculation before finishing the note.
In software, that pause can be a request for a document or a search. You can require the missing evidence before the draft continues.
ReAct research, first submitted in 2022, combined reasoning with actions that gathered observations from tools, including Wikipedia search.
You can use that pattern with a system that has search access, or provide the document yourself when it does not.
Ask it to open the relevant page and check the passage, not merely list a link that looks right.
If the assistant cannot reach the source, you should see that gap. Search can also return the wrong plan or an old page, so inspect what it found.
Looking back is already part of some AI workflows
After correcting the yearly total, the writer replaces the old $144 in the summary with $1,728.
Our guess is that more tools will check important claims during drafting and show you the source behind each correction.
For Sam's note, the tool could check the monthly unit before calculating a yearly cost, then recheck the summary after that number changes.
You can already build a draft-check-rewrite loop. Anthropic described a generator and evaluator workflow in its 2024 engineering guide.
The model can still write each draft one token at a time. The system adds checks between drafts and supplies the results in the next request.
Extra checking takes time and model calls. You need to decide which claims deserve that cost, then measure whether the checks catch actual errors.
Give agent B its own route to the evidence
Sam asks agent A to draft the purchasing note, then gives agent B the original price sheet and the review rules in a fresh conversation.
Before seeing A's total, B calculates the monthly and yearly costs from the source. It then compares those results with A's draft.
You can ask B to return the exact sentence it disputes, the relevant source line and the corrected calculation or wording.
A revises the draft using that feedback. B checks the changed passage and its summary against the original sheet again.
Keep the source available to both roles so you can inspect their claims too.
A multi-agent debate study reported gains on its tested tasks, but also observed confident agreement on wrong answers.
A fresh conversation separates the drafts' context, not the models' knowledge. If you use the same model twice, you can still get the same mistaken assumption.
Keep the disagreement visible until someone checks it
Suppose B flags the annual total and A replies that the price includes a discount. The original sheet never mentioned one.
We would reject that explanation until someone supplies the missing discount terms. You need evidence for the new claim, even if both agents accept it.
If your agreed review budget runs out, keep the disputed sentence marked as unverified and ask a person to settle it.
Do not confuse agreement between two agents with confirmation from a source. You can also compare this process with our small rules-file experiment.
If the source is an API response, our JSON response visualizer can help you inspect its fields. It cannot confirm whether the values are true.
Save the original source and the correction record with the finished note so you can explain why the final total changed.
Try the checklist on one draft today and verify one important claim yourself before you send it.