Skip to content
Working with the tools, not about them

Pitfalls

When checking costs more than doing, the tool has stopped saving anything

Every use has a verification cost attached, and the tasks worth automating are the ones where confirming the answer is cheaper than producing it.

By Maya Chandrasekar3 min read

Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

The comparison that decides everything

There is a simple test that resolves most arguments about where these tools belong. Compare the cost of producing the thing yourself against the cost of producing it this way plus the cost of confirming it is right. When the second total is larger, the tool is the wrong instrument, however impressive the output looks.

The reason this isn’t obvious is that the two costs fall at different moments and often on different people. Generation is instant and belongs to whoever asked; verification is slow, arrives later, and frequently belongs to a reviewer who wasn’t part of the decision.

Applied honestly, this test disqualifies a surprising number of popular uses and endorses some unglamorous ones. That is a feature. Any framework that approves of everything isn’t helping you choose.

Some outputs are cheap to check and some are not

Checking is cheap when the answer can be compared against something that already exists. A translation into a language somebody in the room reads. A summary of a document you have read. Code with tests. An extraction with the source line attached. A calculation you can redo. In all of these, confirmation takes a fraction of the time production would have.

Checking is expensive when confirming the answer requires the same work as finding it. A claim about a field you do not know. A summary of material you have not read. An analysis whose method is not visible. A recommendation about a jurisdiction you have never worked in.

The second list is where the trouble is, and the trouble is worse because those tasks are precisely the ones people most want help with. The appeal of an answer is highest exactly where the ability to evaluate it is lowest, and that correlation is not an accident.

Skipping the check is the decision people actually make

Faced with a verification cost that exceeds the time saved, almost nobody reverts to doing the work themselves. They accept the output unchecked, and the process retains the appearance of a saving because the cost has been converted into risk rather than removed.

That conversion is sometimes reasonable. Low-stakes internal material, a starting point that will be rewritten, one of forty images nobody will study — plenty of work does not warrant verification and saying so explicitly is better than pretending it was checked.

It stops being reasonable the moment the output leaves the room. Anything a customer sees, anything filed, anything that becomes the basis of a decision, anything that will be quoted onwards: the risk has been transferred to someone who did not agree to carry it, and generally does not know they are.

The uncomfortable version is that a great deal of published output has crossed this line quietly, not through anyone deciding to take a risk but through nobody noticing that a decision was available.

Move the task, not the effort

When the arithmetic fails, the productive response is usually to change the task rather than to check harder. Ask for the parts that are cheap to verify. Ask for the search terms rather than the conclusion, the structure rather than the content, the objections rather than the judgement.

Attaching a trace does the same job. Output that carries its sources, its intermediate figures or the quoted line each claim rests on converts an expensive verification into a cheap comparison, and that single change moves a task from the wrong side of the test to the right side.

And where a task genuinely cannot be made checkable, the honest options are to do it yourself, to ask someone qualified, or to accept the risk deliberately and say so. Those are three respectable answers. Quietly hoping is not one of them.

Verification cost falls with repetition

One thing does shift the arithmetic over time. A process you have run two hundred times has a known failure pattern, and knowing where it goes wrong makes checking dramatically faster, because attention can be aimed rather than spread.

This is the strongest practical argument for repeated, narrow uses over occasional broad ones. The same task done regularly builds the knowledge that makes it safe; a different unfamiliar task each week never accumulates any.

It is also why the first weeks of any new use should be checked far more heavily than feels necessary. That effort is not overhead. It is what buys the cheap verification later, and skipping it means never learning where the process fails until something visible goes wrong.

Common questions

How do I decide whether a task is worth it?

Compare doing it yourself against generating it plus confirming it is right. If the second total is larger, the tool is the wrong instrument. The comparison gets skipped because generation is instant and verification arrives later, often for a different person.

Which tasks are cheap to verify?

Those where the answer can be compared against something that already exists — a document you have read, code with tests, a calculation you can redo, an extraction carrying its source line. Expensive ones are those where confirming requires the same work as finding the answer.

What if the checking cannot be made cheap?

Change the task rather than checking harder: ask for search terms rather than conclusions, structure rather than content, objections rather than judgements, or output that carries a trace. Failing that, do it yourself, ask someone qualified, or accept the risk explicitly.

Pitfallsverificationcostjudgementworkflows
Maya Chandrasekar
Senior writer, Prompt After Prompt

Maya writes about prompt craft, writing with ai, images & audio, mostly the parts other people skip and thinks most subjects are more interesting once you know how they work.