An answer drawn from your own document can still come from somewhere else

Supplying the document changes the risk, not the need to check

Asking a question about a document you have provided feels categorically safer than asking an open question, and to a real extent it is. The material is present, the answer can be grounded in it, and the most familiar failure — an invented fact about the world — is much less likely.

What people take from that is a level of trust the arrangement does not support. The answer is not guaranteed to come from the document, and nothing in the output distinguishes a passage that was read from a plausible completion of what such documents usually say.

The practical shift is in which failures to look for. The concern is no longer invention from nothing. It is importation, misattribution and smoothing, and those look different on the page.

Silence in the document is where the trouble starts

The sharpest failure occurs when the answer is not in the material at all. A question about a term the contract does not address, a figure the report does not contain, a policy the handbook never covers. The question presupposes an answer exists, and what comes back is often the answer such a document would usually contain.

This is difficult to catch precisely because the response is reasonable. Most contracts of that type do address the point, most handbooks do have that policy, and the general answer is usually correct in general.

The defence is to ask the question in a form that allows a negative. Ask whether the document addresses the point, and where, before asking what it says, and treat an answer with no location attached as unconfirmed.

It helps to state explicitly that not present is an acceptable response, since a question phrased as what does the contract say about termination carries an assumption that a question phrased as does the contract address termination does not.

Ask where it says that

Requesting the supporting passage alongside the answer is the single most useful habit here, and it is useful for a mundane reason: it converts an assertion into something you can verify in ten seconds by searching the source.

Quoted passages should be checked rather than trusted, since a quotation can be approximate and an approximate quotation of a clause is a different clause. Search the document for the quoted string. If it is not there verbatim, you have learned something important about the answer that contains it.

Locations matter as much as quotations. An answer citing a section that exists, with wording that does not appear in it, is a specific and common failure and it is invisible to anybody who does not go and look.

Several documents get reconciled without a note

Supply a set of documents and a new failure appears. Where two of them disagree — an older policy and its replacement, two reports with different figures, a contract and the side letter that varies it — the answer tends to arrive as a single coherent statement.

The disagreement is usually the thing you needed to know. A superseded version quietly blended with the current one produces an answer that is not true of either document, and it reads as more authoritative than a straightforward quotation from one of them.

So ask about consistency directly. Whether the documents conflict, where, and which is more recent, is a better first question than anything about the substance when the material comes from more than one place.

It also pays to supply less. A set trimmed to the documents that are actually current removes most of the opportunity for this failure, and deciding what is current is a task for the person who knows the history rather than for anything reading the files.

Match the checking to what the answer is for

This is not an argument for verifying every answer about every document, which would eliminate the benefit. Reading a long report to find which section discusses a topic is low-stakes, easily corrected and enormously useful, and treating it with suspicion is a waste of effort.

The care belongs where the answer will be relied on: an obligation in a contract, a figure entering a decision, a clinical or regulatory requirement, anything that will be quoted to somebody else. For those, the answer is a pointer to the passage you should read yourself.

And where the document is the whole basis of a consequential decision, the honest position is that a summary of it is not a reading of it. Use the tool to navigate the material and then read the part that matters, which is a smaller job than reading everything and a much larger one than accepting a paragraph.