Skip to content
Working with the tools, not about them

Prompt Craft

Rewrite the question until the answer is one you could check

Where you have no way to verify what comes back, the practical move is to change the question rather than to trust the answer more.

By Naina Sethi4 min read

Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

Unverifiable questions arrive constantly

A great deal of what people ask sits outside their ability to confirm. What does this regulation require in a country you have never worked in. Is this code doing what its author claims. What does the specialist literature say about a treatment. What is this contract clause for. In each case an answer arrives, fluent and plausible.

The honest position is that you cannot tell whether it is right, and no amount of confidence in the phrasing changes that. Fluency and accuracy are produced by the same process here, so the smooth answer and the wrong answer look identical from where you are sitting.

Deciding to trust it anyway is a choice with consequences you will meet later. Deciding not to use the tool at all is often over-correction. There is a third move, and it is the one that gets used least: change what you asked for.

Ask for the parts you can check

A question can nearly always be decomposed into pieces of different verifiability. The claim itself may be unverifiable, but the vocabulary isn’t, the structure of the problem is not, and the list of things a competent person would look at is not. Those are checkable in minutes.

So instead of asking what the rule is, ask what the rule is called, which body issues it, what the related terms are, and what a practitioner would search for. You now hold a set of search handles that lead to a primary source, which is where the answer was always going to have to come from.

This inverts the usual relationship. The tool stops being the source and becomes the index, and an index that is occasionally wrong is far less dangerous than an authority that is occasionally wrong, because you were going to open the real document anyway.

Ask what would make the answer false

A useful second pass is to ask what the answer depends on and what would break it. Under what conditions does this not hold, what would have to be true for it to be wrong, what is the common exception. The answers tend to be more reliable than the original claim.

This works because conditions and exceptions are structural rather than specific. Getting the exception category right requires less precision than getting a figure or a citation right, and the categories give you something concrete to check against your own situation, which you do know.

It also surfaces the cases where the question was malformed. If the answer to what would make this wrong is a long list of ordinary circumstances, you have learned that the confident paragraph you received was describing an idealised case that has little to do with yours.

Cross-checking has real limits

People often try to verify by asking again, in another session or another tool, and treating agreement as confirmation. It is weak evidence. Two answers can be wrong in the same way for the same reason, and the agreement then reads as corroboration when it is nothing of the kind.

Disagreement is the more informative outcome. If two attempts contradict each other, at least one is wrong and you have learned that the question isn’t as settled as the first fluent paragraph implied. That is a genuine finding, and it is worth the second attempt on its own.

Agreement, though, tells you mainly that the question is common. It should raise your confidence a little and should never be the last step for anything that carries a cost when it turns out to be wrong.

Some questions should leave the tool entirely

Where being wrong has legal, medical, financial or safety consequences, the right move isn’t a better prompt. It is a qualified person, a primary source, or a test you can run. That is not a limitation of technique — it is what the word unverifiable means when the stakes are real.

The tool still earns its place before that conversation. Arriving with the correct terminology, a sense of the structure of the problem and three specific questions makes an hour of expensive advice go considerably further than arriving with a vague description of your situation.

That is a decent summary of the whole approach. Use it to sharpen what you ask a person, a document or a test, and reserve your trust for the thing that can actually be checked.

The habit that follows is a small one and it holds up well. Before accepting anything you could not have produced yourself, ask what would have to happen for you to find out it was wrong, and how long that would take. If the honest answer is that you would never find out, you are not using an answer, you are adopting one.

Common questions

Does asking the same question twice help?

Only asymmetrically. Contradiction between two answers is real evidence that the question is unsettled. Agreement is weak evidence, since two attempts can be wrong in the same way for the same reason, and the matching answers then feel like confirmation without being it.

What is safe to accept without checking?

Vocabulary, structure, and the names of things — the material that gives you search handles rather than conclusions. Those are cheap to confirm the moment you open a real source, and they are what most usefully speeds up the checking that follows.

Is it worth asking about a field I know nothing about?

Yes, for orientation, and not for conclusions. Coming away with the right terms, the shape of the problem and a short list of specific questions makes the subsequent conversation with a qualified person far more productive.

Prompt Craftverificationquestion designaccuracyresearch
Naina Sethi
Consumer editor, Prompt After Prompt

Naina covers prompt craft, writing with ai, images & audio and the questions readers actually send in and is happiest when a piece answers the question completely.