Catching a wrong answer means checking the claim, not asking again

Errors concentrate in predictable places

The useful starting point is that wrong answers are not evenly distributed. They cluster around anything specific and thinly documented: an individual case reference, a version number, the terms of a particular scheme, a page in a book, a date in a niche field, an attribution of a phrase to a person.

Broad explanations of well-covered subjects are comparatively safe, which is why the experience of using these tools feels reliable — most casual questions fall into the safe category. The moment a question narrows to a particular, the ground changes underneath without any change in tone.

This gives you a triage rule that costs nothing. Read your own output looking for the specifics, and treat every one of them as unverified until it has been checked against something outside the conversation. Everything else can usually wait.

The techniques that stay inside the conversation are filters

Asking whether the answer is correct, requesting a critique of it, running the question again in another session and comparing, asking for a confidence rating: all of these do something, and none of them is verification. They are correlated with correctness weakly enough that a confident wrong answer will frequently survive all four.

Consistency across separate runs is the least bad of them. An answer that comes back the same way three times from independent attempts is somewhat more likely to be well supported than one that varies, because variation suggests the material was thin. Somewhat is doing a lot of work in that sentence.

Confidence ratings deserve particular scepticism. A number attached to a claim by the same process that produced the claim is another piece of generated text, and it will be well calibrated in some places and not in others with no way to tell which from the number itself.

Design the request so checking is possible at all

A great deal can be done before the answer arrives. Supply the source material rather than asking for recall, since an answer drawn from a document you provided can be checked against that document. Ask for the supporting passage to be quoted alongside each claim. Ask for anything not covered by the source to be listed separately rather than filled in.

Quoted passages are especially useful because a quotation is mechanically checkable: search the source for the string and it is either there or it is not. That single technique converts the most expensive kind of verification into something that takes seconds and can be scripted.

The same logic applies to references. Asking for citations without checking them is worse than not asking, because a list of plausible references creates the appearance of grounding. If you ask for them, look them up; if you are not going to look them up, do not ask.

Regenerating is not investigating

The common reflex on receiving something that looks wrong is to ask again, perhaps with more emphasis. This produces a different answer, which feels like progress and tells you nothing about which one is right. Two conflicting outputs with no external reference leave you exactly where you began, minus some time.

It is also how a correct answer gets discarded. Pushed to reconsider, a tool will often revise a right answer into a wrong one, because agreement with the person expressing doubt is a strong pull. Persistence in questioning is not a truth-finding procedure here.

The productive move at that moment is to leave the tool and check the specific claim at its source. Usually this takes a couple of minutes, and it ends the question rather than generating another version of it.

Proportion, and the honest cost

None of this means checking everything to the same depth. The proportionate approach is to spend attention where an error would be expensive or hard to reverse, and to accept unchecked output where a mistake is cheap, visible and correctable. A draft for a colleague and a figure entering a published report warrant entirely different treatment.

What should be resisted is the middle position where verification is nominally part of the process but consists of reading the output and finding it reasonable. Reasonableness is what the tool produces by construction, so that check is measuring the wrong property.

And there is a real cost worth naming. On some tasks, thorough checking takes long enough that the tool has saved nothing. When that is true it is a finding about the task rather than a failure of technique, and the useful response is to stop using it there rather than to check less.