Skip to content
Working with the tools, not about them

Pitfalls

Material that has already been through the tool is a poor input for the next step

Generated text fed back in as source loses its remaining specifics with each pass, and the loss is invisible because every version reads as competently as the last.

By Adrian Novak3 min read

Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

Each pass moves the text towards the average

Rewrite a paragraph, then rewrite the rewrite, then summarise that and expand it again. Every stage is defensible on its own and the direction of travel is consistent: specifics thin out, hedges accumulate, unusual phrasings regularise, and the piece drifts towards the most ordinary version of what it is about.

What survives longest is structure and what disappears first is detail, which is the opposite of what you want. After three or four passes you have a well-organised document whose distinguishing content has quietly evaporated, and there is no version of it that looked obviously wrong along the way.

The absence of an alarm is the whole difficulty. Text doesn’t degrade visibly in the way a repeatedly copied image does. Each generation is as fluent as the one before, so fluency cannot be used to detect the loss.

It happens without anyone deciding to do it

Almost nobody sets out to run material through repeatedly. It arrives through ordinary work: a draft is generated, the summary of it is generated from that draft, a colleague expands a section of the summary, and the resulting document is used as background for the next piece.

Multi-step workflows produce the same pattern by design. A pipeline that extracts, then summarises, then drafts is three passes over material that has only ever been seen directly by the first step, and everything after it is working from a description of a description.

Long conversations do it as well. Twenty exchanges of refinement over the same passage is a chain of rewrites, and the text at the end may have very little contact left with the source it began from.

Return to the source, not to the last version

The correction is straightforward and requires discipline rather than technique. When a piece needs another round, go back to the original material — the notes, the source document, the transcript, the data — rather than to the most recent output.

This costs a little more each time and it stops the drift entirely, because every pass is measured against something fixed. It also tends to catch what was lost two versions ago, which no amount of editing the current draft could recover.

Keep the source available, then. A workflow that discards its inputs after the first step forces every later stage to work from generated material, and it is worth noticing that this is a design decision rather than an inevitability.

The same applies to a long refining conversation. When it stops improving, take what you have back to the original brief and start a clean pass rather than continuing to sand the same surface.

External material may already have been through it

The version of this problem you cannot see concerns sources from outside. A page, a summary, a product description or a set of notes handed to you may itself be several passes from anything anybody observed, and there is no reliable way to tell by reading it.

This argues for the same preference a careful researcher has always had: prefer material closer to the source. A primary document, a recording, a dataset, a person who was there. Not because generated material is worthless but because you can see how far you are from the origin.

It also argues for being explicit about your own outputs. Marking a document as drafted from these notes, or as a summary of that report, costs a line and tells the next person how many removes they are working at.

Where circulating material back through is legitimate

None of this rules out iteration. Passing a draft back for a specific bounded operation — check these figures against the source, apply this style rule, find the passages that lack support — isn’t the same as regenerating it, because the text is being examined rather than rewritten.

The distinction is whether the operation is allowed to produce new prose. Editing operations with a stated scope do not drift. Open rewrites do, every time, and the difference between the two is entirely in how the request was framed.

So the practical rule is narrow and easy to apply. Rewriting works from the source; checking and correcting can work from the draft. Confusing the two is how a piece with real material in it turns into a piece about the subject in general.

Common questions

What actually degrades across repeated passes?

Specifics. Structure and fluency survive while details, unusual phrasings and qualifications thin out, so the document becomes a well-organised version of the ordinary account. Nothing about the text signals the loss, because each pass reads as competently as the last.

How does this happen without anyone intending it?

Through ordinary sequences — a draft, a summary of the draft, an expansion of the summary — and through multi-step pipelines where only the first stage ever sees the source. Long refining conversations are the same chain in another form.

Is it ever fine to work from a generated draft?

Yes, for bounded operations that examine rather than rewrite: checking figures against the source, applying a style rule, finding unsupported passages. Anything permitted to produce new prose should start from the original material instead.

Pitfallsinputsdraftingsourcesquality control
Adrian Novak
Deputy editor, Prompt After Prompt

Adrian has written about prompt craft, writing with ai, images & audio for most of the last decade and prefers a plain explanation to a clever one.