In May 2026, a UK High Court judge sat down to deliver his ruling in a routine insolvency case and used the occasion to hand down something closer to an indictment. A junior solicitor at one of the UK's largest and most respected law firms had used AI to draft a letter citing a rule from the Insolvency Rules. The judge checked the citation himself. The wording didn't exist anywhere in the actual legislation. He called the failure to check it before filing "inexcusable."
What happened next is the detail that matters. The firm then used AI again — this time to draft a letter explaining the original mistake to the court. That letter was also wrong. The judge concluded the firm had misled the court twice, using the same tool both times, with no person catching either error — not, he was careful to note, through any deliberate attempt to deceive, but through the simple absence of anyone checking. The firm referred itself to the Solicitors Regulation Authority, and the judge took no further action.
This usually gets filed under a familiar headline: another AI hallucination story, another cautionary tale about a technology that isn't ready yet. That's the wrong read. The tool did exactly what it was built to do — generate plausible language at speed, twice. The firm's model didn't fail. Nobody, on either pass, stopped to check it.
There's a phrase for this that predates the transformer, the neural network, even the computer as we know it, by more than a century. In 1864, Charles Babbage wrote about a question he kept being asked regarding his mechanical calculating engine: if you put the wrong figures in, will the right answer come out? He couldn't quite understand the confusion behind the question. Nearly a hundred years later, an IBM instructor named George Fuechsel gave that confusion a name for a new generation of students: garbage in, garbage out. Computers, he told them, only process what they're given. Feed it bad data, and it will hand you back a bad answer.
This has nothing to do with artificial intelligence specifically. What matters is what happens when a system executes instructions without judging them — which is what every computational system, from Babbage's engine to the model that drafted the firm's letters, was built to do. The machine was never the one skipping the review. A person was.
Go back further than computing and the same idea shows up wearing different clothes. Traditional book publishing built an entire chain of professionals whose sole function was friction: an acquisitions editor who rejects a manuscript for lacking merit, a developmental editor who forces a rewrite of a weak structure, a proofreader who halts production over a typo at the final proof stage. These people don't exist to slow authors down for its own sake — they exist because a writer's judgment about their own writing is often the least reliable judgment in the building.
That friction was never really about time — it was outsourced taste, other people's judgment standing between a creator's first instinct and the public, catching what a writer was too close to their own work to see.
Solo creators — bloggers, YouTubers, and by extension nearly anyone producing anything alone at a laptop — never had that chain. Most compensate for its absence deliberately: a rough cut sent to a peer for a brutal note, an editor hired specifically to be the person who says no, audience retention data treated like a stand-in producer.
What AI took from solo creators wasn't some new kind of friction — it was the last bit they had left. The habit of pausing, rereading, asking whether this is actually good — the closest thing most people have to their own personal copyeditor — is exactly the step that generating a finished-looking paragraph in four seconds makes easiest to skip. Nothing about the output announces that no one checked it. It reads exactly as fluently as work someone actually stood behind.
Multiply this by every desk producing decks, code, reports, and filings, and this stops looking like an anomaly and starts looking like a preview — and not just in the UK. In the first quarter of 2026 alone, courts in the US imposed more than $145,000 in sanctions against lawyers for fabricated AI citations — over $100,000 of that against a single lawyer in one case, plus the first serious penalty a US appeals court has ever handed down over an AI-generated filing. Different jurisdiction, same failure, same absence of a person in the loop.
The damage isn't confined to law, or to one country. Admiral, one of the UK's largest insurers, reported a 71 percent year-on-year surge in detected fraud in 2025 — fraudsters using generative tools to fabricate accident photos, damaged vehicles, and supporting documents convincing enough to slip past automated claims systems built assuming a human had touched the file. The people absorbing that cost aren't the fraudsters. They're the ordinary policyholders paying the resulting premium increases.
And it compounds in a direction most people producing content for these systems to read haven't fully priced in. Research on model training has shown that when generative systems are trained repeatedly on their own unreviewed output, quality degrades in a measurable, compounding way — researchers now call it model collapse. What gets published without a second look today doesn't just embarrass the person who published it. It becomes raw material for whatever gets generated next.
It would be easy to read all of this as a technology story, and that reading is exactly why it keeps happening. Somewhere in every one of these cases is a version of the same sentence: it hallucinated. Passive voice, no author. As if the model were an intern who went rogue rather than a very fast, very literal instrument doing precisely what it was pointed at.
Go back to Babbage's question. If you put wrong figures in, will the right answer come out? The confusion he couldn't quite understand people having in 1864 is the same confusion still driving the coverage today. A model doesn't have opinions about the quality of what it produces. It has no internal editor quietly asking whether this is actually good. That job — the only job that was ever actually yours — is the one step nobody in these stories appears to have performed. The output isn't a verdict on the tool. It's a fairly accurate transcript of how much attention you paid.
Say the word taste out loud in a work meeting and watch people's eyes glaze — it sounds like a personality trait, something you either have or don't, useless as instruction. It isn't. Break it down and it's a set of specific, learnable habits.
A vocabulary for what's wrong is one piece of it. A trained designer doesn't just feel that a layout is off; they can name the exact spacing decision causing it and the fix for it. An untrained eye gets the feeling and stops there, with no route from noticing to correcting.
Subtraction is another. Skilled work is defined as much by what's removed as by what's added — noise cut for hierarchy, decoration cut for clarity. Untrained instinct runs the other way: when something feels incomplete, the reflex is to add another slide, another paragraph, another qualifier, until the point is buried under its own decoration.
And fluency with constraints matters just as much. A tailor who understands how fabric drapes on the bias isn't fighting the fabric; a product designer who understands an edge case designs around it instead of being surprised by it later. Craft converts a trade-off from an obstacle into working material.
None of these three arrived with AI, and none are optional extras layered on top of using it well. They're the entire difference between someone who reviews five outputs, keeps the one worth keeping, and can say exactly why — and someone who takes the first output because it looks finished, with no internal basis for knowing whether it actually is.
There's a narrower, more practical version of this specific to working with these tools, and almost nobody gets taught it: context management — the discipline of deciding what information, constraints, and prior work actually belong in front of a model before asking it for anything. It's arguably the single most consequential skill in working with AI well, and it barely exists as a taught subject. Most people skip straight to typing a request into an empty box and treat whatever comes back as an answer, rather than as a first draft from an assistant who knows nothing about the actual situation unless told.
Which is really the whole argument compressed into one habit: prompting well isn't a technical skill sitting next to taste — it's taste itself, applied to a new material. Someone with a real internal standard treats a model's output the way an editor treats a first draft — as raw material to interrogate, cut, and mostly discard. Someone without one treats it as the finished thing, because they have no standard of their own to check it against, and the output, fluent and confident by default, gives them no reason to suspect otherwise.
That's the actual gap between those filings and the version of them that never makes the news: not a better model, not a more advanced tool. A person, somewhere in the process, who read the output and asked whether it was actually right before it went out the door.
Widen the lens far enough and this stops being a story about slides and legal filings at all. Marshall McLuhan spent a career arguing that we shape our tools and our tools, afterward, shape us: the printing press produced what he called typographic man; broadcast media produced the inhabitant of what he named the global village. It isn't unreasonable to guess that generative AI is shaping something in turn — a kind of person who navigates the world less through fixed certainty and more through a continuous stream of updated predictions and generated possibilities, checked or not.
The deeper shift McLuhan's frame points toward isn't that machines are getting smarter. It's that humans may be quietly changing what they think knowledge even is — from something you hold, verified and fixed, to something you dip into, provisional and always regenerating. In that world, gathering information stops being the scarce skill. Everyone has access to infinite information now, instantly, for free. What's actually scarce is the judgment to look at what's been generated and decide, correctly, what's worth keeping.
So the real question was never how to write a better prompt. It's this: everyone can produce infinite output now. What's still scarce is someone willing to stand behind any of it — starting with you and the work you produce.