When an AI feature underperforms, teams tend to reach for a different model or rewrite the prompt for the fifth time. Neither usually helps, because the real fault sits in the context—the information assembled and handed to the model at request time. The encouraging part is that context failures are not random. The same handful of mistakes show up again and again.
This article names seven of the most common, drawn from patterns that repeat across teams of every size. For each one you will find what it looks like in practice, why it happens, what it costs, and the specific correction. The goal is recognition: once you can spot a failure mode by its symptoms, fixing it becomes routine.
Read these as a diagnostic checklist. The next time an answer comes back wrong, run through the list and you will often find the culprit before you have changed a single word of your prompt.
Mistake 1: Dumping Everything Into the Window
The instinct to include more information feels safe. It is not.
Why It Happens
Large context windows make it tempting to paste entire documents, full histories, and every loosely related file, on the theory that the model will sort it out.
The Cost and the Fix
Irrelevant text dilutes the signal the model needs and raises cost on every call. Accuracy often drops as volume rises. The fix is ruthless selection: include only material that could change the answer. A focused set of relevant passages beats a comprehensive dump every time. The discipline behind this is covered in Master Context Engineering Without Guesswork.
Mistake 2: Ignoring Position
Where information sits in the context is not neutral.
Why It Happens
Teams treat the context as an unordered bag, assuming the model reads everything equally.
The Cost and the Fix
Models attend most strongly to the beginning and end of the window. A critical rule buried in the middle gets overridden by surrounding text. The fix is deliberate ordering: place non-negotiable instructions at the edges and restate the immediate task right before generation.
Mistake 3: Treating Retrieval as Solved
Retrieval quality sets the ceiling on answer quality, yet it is often the least examined part.
Why It Happens
Once a retrieval pipeline returns something, teams assume it returned the right thing and move on.
The Cost and the Fix
If retrieval surfaces the wrong passages, no prompt can rescue the answer—the model is grounding on bad evidence. The fix is to inspect what retrieval actually returns for failing cases and tune the lookup before touching anything downstream. Tool choices that affect this are compared in Survey the Tooling Landscape for Context Engineering.
Mistake 4: Letting Conversation History Rot
Multi-turn experiences accumulate context that quietly degrades.
Why It Happens
The simplest implementation appends every message forever, which works fine in early testing.
The Cost and the Fix
Eventually history overflows the window, drops the system instructions, or buries current intent under stale exchanges. The fix is active management: replace old verbatim turns with running summaries so intent survives while the window stays lean.
Mistake 5: Allowing Context Poisoning
Once a wrong fact enters the context, the model can treat it as gospel.
Why It Happens
A hallucinated detail, an outdated record, or an erroneous tool result gets fed back into later context and compounds.
The Cost and the Fix
The model builds on the bad information, producing confident, consistent, and wrong answers. The fix is to validate what enters context, especially model-generated and tool-returned content, and to remove poisoned material rather than reasoning around it. More failure cases like this appear in Context Engineering: Real-World Examples and Use Cases.
Mistake 6: Vague System Instructions
A rule the model cannot act on is not a rule.
Why It Happens
Instructions like be accurate or be professional feel responsible but carry no concrete constraint.
The Cost and the Fix
The model has nothing specific to obey, so behavior drifts. The fix is concrete, testable instructions: cite only the provided sources, respond in under three sentences, never recommend a competitor. Specificity turns intent into enforceable behavior. A good test is whether you could write a check that confirms the rule was followed. If you cannot imagine such a check, the instruction is too vague for the model to reliably honor.
A Related Trap: Contradictory Rules
As instruction sets grow, rules begin to conflict—one says be thorough, another says be brief. The model cannot satisfy both and picks unpredictably. Review your instruction set as a whole, not just line by line, and resolve contradictions before they produce inconsistent behavior you cannot explain.
Mistake 7: Shipping Without Evaluation
Without measurement, every change is a guess.
Why It Happens
It works on my example feels like enough, especially under deadline pressure.
The Cost and the Fix
One passing example hides the many cases that fail, and later changes silently break earlier wins. The fix is a regression set of real cases that every change must pass. Failures get traced to context first, fixed, and added to the set. The full workflow is laid out in Build Reliable Context One Step at a Time.
How These Mistakes Compound
The mistakes above rarely appear alone, and they reinforce each other in ways that make diagnosis harder.
One Fault Masks Another
A team that overstuffs the window and also positions a rule poorly may fix the positioning, see no improvement, and wrongly conclude positioning did not matter—when in fact the noise from overstuffing was drowning the now-correct rule. Fixing one fault at a time, verifying each against a regression set, is what keeps these interactions untangled.
Why Evaluation Sits at the Center
Notice that the final mistake—shipping without evaluation—is what makes all the others hard to catch. Without a regression set you cannot tell which fix helped, which made things worse, or whether a fix held. Evaluation is not just one more item on the list; it is the practice that turns every other correction from a guess into a verified improvement.
Frequently Asked Questions
Which of these mistakes is the most common?
Dumping everything into the window is the most widespread, because large context windows make it feel free and safe. It rarely is. Overstuffed context dilutes the signal the model needs and quietly raises cost on every single request while often lowering accuracy.
How do I tell if retrieval is my problem?
Take a failing case and read the exact passages retrieval returned for it. If the right facts are not there, retrieval is your bottleneck and no prompt change will fix it. This inspection takes minutes and resolves a large share of mysterious wrong answers.
Is context poisoning really a risk if I do not feed model output back in?
Even without an obvious loop, stale retrieved data and erroneous tool results count as poisoning. Any wrong fact that enters the context can be treated as authoritative. Validating what enters context—not just model output—is the broader safeguard.
Why are vague instructions so common if they do not work?
They feel responsible and are easy to write under pressure. The problem is that the model cannot act on an abstraction. Converting be accurate into a concrete, testable rule takes more thought but produces behavior you can actually verify and enforce.
How small can my evaluation set be and still help?
Even five to ten realistic cases catch most regressions. The value is in covering easy, typical, and adversarial inputs, and in growing the set every time a real failure surfaces. A small, living test set beats an exhaustive one you never run.
Key Takeaways
- Overstuffing the window dilutes signal and raises cost; select ruthlessly
- Position matters—place critical rules at the high-attention edges
- Retrieval quality caps answer quality, so inspect what it actually returns
- Manage conversation history with summaries to prevent overflow and drift
- Guard against poisoned context from stale data and erroneous tool results
- Write concrete, testable instructions and verify every change against a regression set