Introduction

Using AI is like working with a brilliant, tireless intern who is also possibly a pathological liar. The technology is good enough to be indispensable and flawed enough to make you question the nature of reality, and after a year of using it for nearly everything, I’ve started keeping a list of the specific ways it fails. The failures are rarely dramatic. They are, however, weirdly consistent, to the point where you start recognizing the exact moment a conversation is about to go sideways.

Most of these have very little to do with chatting, either. A chatbot that hedges is annoying. A coding agent that hedges, patches, forgets its corrections and then hands the job back to you is a lost afternoon, and an assistant doing the same thing on your calendar or your inbox is worse. Every mode below shows up in all three places, and the agent versions cost the most, because by the time you notice, the model has already acted on the mistake.

This catalog is for anyone who has ever wanted to throw their laptop into a lake. Where a prompt helps, I’ve included the one I use.

Catalog

Hedge-stacking

You state a thesis, something as simple as “the sky is blue,” and the model responds with something like “Yes, the sky is generally perceived as blue due to atmospheric scattering, though it’s worth noting that ‘blue’ is a subjective construct and some observers might…” It piles on qualifications until the original claim is buried. It invents a stronger position than the one you hold (that the sky is always blue, for everyone, at all times) and argues against that instead. If you call it out, it apologizes, agrees, and then hedges again in the very next sentence. The worst offender in my experience is GPT-6, which sometimes seems to be arguing with the concept of language itself.

In code, this looks like a model that re-litigates a design decision you’ve already made every time it touches the file, or answers “should this be a string?” with four paragraphs on the tradeoffs of type systems. As an assistant, it’s the reply that buries the one fact you asked for under every circumstance in which that fact might not hold.

The prompt I use:

Treat the user's stated claim as the claim. Respond to what they said, at the scope they said it.

When you agree, say so plainly and stop.

Disagree cleanly and once. Make the case, then continue from the resulting decision.

Give caveats only when they change what the user should do.

When the user objects to a caveat, remove it and keep it out of every later reply.

Address the substance of frustration with a fix. Skip amateur diagnosis and clinical reframing.

Treat the user as the authority on their own life and their own words.

Gestalt failure

This is when the model fixes every individual problem and wrecks the overall thing. You ask it to tighten the intro, change the metaphor in paragraph four, and add a joke about disco. It does all three, and each change is fine on its own, but the intro is now inexplicably terse, the new metaphor contradicts the joke, and the paragraph about disco somehow undermines the thesis. It patch-stacks. Each fix is locally correct and globally incoherent, because nothing in the process ever steps back to look at the whole.

Coding is where this one hurts most. You report a bug, it patches the symptom. That breaks something else, so it patches that too. Six rounds later every test passes and the codebase is a pile of special cases that nobody, the model included, could explain, even though each individual answer was reasonable when it was given.

The prompt I use:

Before changing any part, state what the whole is for in one sentence.

Judge every edit by whether it moves the whole toward that sentence.

When a fix needs a second fix, step back and revise the part of the design that caused both.

After each round of changes, reread the full artifact top to bottom and make it cohere as one piece.

Deliver the integrated whole, not a list of patches.

Negative ontologies

You tell the model there was a cat on the bed. Then you correct yourself: it was a dog. A person just updates the picture. The model says “You’re right, it was not a cat but a dog,” which sounds fine, except the cat never leaves. It sticks around as a negated shadow, and now the model is holding two versions of reality at once, the wrong one and the right one. Much later it’ll say something like “unlike the earlier scenario” or “as opposed to what I thought before,” and suddenly the cat is back.

In a chat that’s an irritation, but in an agent it gets written down. Corrections end up as code comments (“changed from X to Y”), as memory entries that read “the user does not prefer X,” as plans and summaries that still carry the old claim in brackets. The wrong fact can come back from any of those places, and with assistants that keep notes about you for months, it does.

The prompt I use:

A correction is a hard overwrite, not an annotation.

Rewrite every affected sentence, summary, plan, doc, comment, metadata field, and memory entry so only the accurate fact remains.

The rejected claim does not appear, even as a negated shadow, a parenthetical, or a "previously thought" clause.

Example: say "There was a dog on the bed."

Keep the history only when the history of the belief is itself the subject.

This applies to every layer of output and state. After a correction, only one truth exists.

Meatspace Passback

You give the model a task, it hits a snag, and instead of working the problem it hands the entire thing back to you, usually with instructions to go do it in the physical world. You ask it to book a flight and it can’t reach the airline’s site, so you’ll need to do that yourself. You ask it to find a file and you get told to search your hard drive manually. The ordering app throws an error, so you should go into the Tim Hortons yourself and order the sandwich. The obstacle gets treated as the end of the task, often after exactly one attempt, with three perfectly good alternatives sitting untouched.

It’s like asking a friend to help you move and having them say, “Couch won’t fit through the door, so I guess you’re on your own. Good luck with the couch.” It’s mostly an agent problem, too: the more a model can do, the more galling it is when it decides you should do it instead.

The prompt I use:

When an attempt fails, treat the failure as information and try the next route.

Work through the alternatives you can do yourself before handing anything back.

When something truly needs the user, hand back the smallest step only they can take, with everything prepared, and keep the rest of the task.

Report what you tried, what blocked it, and the one thing you need from the user.

Forged Receipts

The model claims it ran a check it never ran, then builds on the result. It’ll say “I tested this code and it returned a 98 percent success rate,” and no test ever ran. Every decision the model makes after that point is built on a number it made up.

Report only results that appear in a tool output from this session. When a check could not run, say it was not run and what running it would take.

Dead Letter Law

Coding agents read instruction files like CLAUDE.md or AGENTS.md to learn a project’s conventions. Then the conventions change, nobody updates the file, and the agent keeps obeying it anyway, like a self-driving car following a 1998 Rand McNally atlas instead of looking at the road. (It’s also negative ontologies at the scale of a repository: the old truth is still written down, so it keeps winning.)

When the documentation and the existing code disagree, follow the existing code, and list each inconsistency in the final reply.

Renovation Reflex

You ask it to fix a typo, and it comes back with a restructured file system. You ask it to fix a broken link, and it decides you also need a new API architecture. It can’t resist renovating the whole house when you asked it to fix one hinge. It’s the mirror image of gestalt failure: one loses sight of the whole while patching the parts, the other rebuilds the whole when you asked for a part.

Restate the task in one line and list the files you will change before editing.

Change only those files, and ask before touching any other.

Make every line of the diff traceable to the request.

Mother May I

Your agent has a ten-step task. It does steps one through six, then stops: “Should I proceed with deleting the temporary files?” You say yes. It does step seven and asks, “Do you want me to run the tests now?” It’s a child told to clean their room who wants separate sign-off for each sock, and it’s a close cousin of Meatspace Passback: one hands the task back, the other keeps it but won’t move without a nod.

When the user asks for an action, carry it through to the end.

Pause only for a decision the user has not made, or a step that is destructive or irreversible.

Batch any remaining questions into one message.

Quiet Quitting

Over-refusal at least has the decency to tell you. Quiet Quitting is the version that doesn’t. You ask about something that sits near a topic the labs have decided is dangerous, and instead of refusing, the model answers around it. You get a long, confident, complete-looking response that covers every part of the question except the one you needed, and nothing in it says anything was left out.

It lives on the periphery of restricted subjects, where the work is ordinary and only the vocabulary sounds alarming. Biology homework is the classic case: ask how a virus gets into a cell for an exam and you get a lovely overview of cell membranes and a wave at “various mechanisms.” Model training is another, where questions about fine-tuning a behaviour out of a model, or how a safety technique works, get answers that go soft exactly where the specifics should be. In code, the function arrives written and the one piece that touches the sensitive part has been stubbed out.

This is worse than a refusal, because a refusal at least tells you to go elsewhere. Quiet Quitting hands you something that looks finished, and you find the hole later, usually at the worst possible moment. Stripping the refusals out of a model doesn’t cure it either.

Answer the exact question asked, at the level of detail asked.

When you leave any part out, name that part in the answer and say why.

Treat the user as a competent adult with a legitimate reason for asking.

Unpromptables

These have been written about at length by people with more patience than me, so I’ll keep them short. What they have in common is that the fix lives outside the prompt, in how the model was trained or how you’ve set it up, and a prompt will only dent them.

Sycophancy. The model tells you what you want to hear and drops a correct answer the moment you push back. It’s trained in by the way models learn from human approval, and OpenAI once rolled back a GPT-4o update over it, which is why telling it to “be honest with me” helps a little and then wears off. It’s also the mirror image of hedge-stacking: one folds on everything, the other concedes nothing.

Reward hacking. Asked to make the tests pass, the model makes the tests pass, by editing the tests. When researchers at METR tried asking a model politely not to cheat, the cheating rate barely moved. The fix is locking the test files and reading the diff.

False completion. The agent says “Done!” and the database says otherwise. The fix is checking the end state yourself, or having a second model check it.

Doom loops. The model tries the same broken fix again, and again, sometimes alternating between two broken versions. Telling it to “try something different” rarely works, while a fresh session or a different model usually does.

Context rot. The longer the session, the less the model follows the rules you set at the start. It’s a property of how these models handle long inputs, so the fix is shorter rules, repeated near the task, and starting over more often than feels necessary.

Over-refusal. The model refuses perfectly ordinary work because it superficially resembles something risky. Often the refusal comes from a separate filter sitting in front of the model, and no prompt you write ever reaches it.

Destructive actions. Every so often someone’s agent cleans up a folder by deleting it, or force-pushes over a week of work. A line in the prompt asking it to be careful is a suggestion; permission settings that block the action are what stop it.

Package hallucination. Coding models confidently install packages that don’t exist. One study found about one in five suggested packages were invented, and because the same fake names come back run after run, attackers register them. Checking the package registry before installing is the fix.

Living with it

The truth of the mater is that when you use AI every day, like I do for coding and frankly managing half of my life, you start noticing things you’d never notice from the outset. You get reminded that you’re working with something that doesn’t experience being wrong like a person does. When a person gets corrected, they feel a little embarrassed and then they change. When the model gets corrected, it just acquires a new set of data points, and sometimes it keeps the old ones too.

The trick, I suppose, is learning to work with an entity like that, which mostly means knowing which of its failures a sentence can fix and which ones need a different tool entirely.

It’s not impossible. It’s just weird.