Extraction failures are insidious because the output almost always looks correct. A model returns a clean record with every field populated, and nothing about it signals that the invoice number is fabricated or the date is in the wrong format. The errors hide in plain sight, which is why they accumulate silently until someone runs a report and the numbers do not add up.
This article names seven specific failure modes that recur across nearly every extraction project. For each one, the goal is the same: explain why it happens, make the cost concrete, and give you the one practice that prevents it. These are not theoretical risks. They are the problems that turn a promising prototype into a data-quality incident, and every one of them is avoidable once you know to look for it.
The pattern across all seven is that the model is doing exactly what it was built to do, which is produce plausible text. The mistakes come from expecting it to behave like a database when it behaves like a writer. Once you account for that, the corrections become obvious.
Mistake 1: Writing the Prompt Before the Schema
Teams often start typing instructions before deciding what the output record should contain, then bolt on fields as they go.
Why It Costs You
Without a defined schema, field names drift between runs, types are inconsistent, and downstream code breaks. The corrective practice is to define every field with a name, type, and required flag before writing a single line of prompt. The schema is the contract; write it first.
Mistake 2: No Rule for Missing Values
When a field is absent, an unguided model fills it with a plausible invention rather than reporting the gap.
Why It Costs You
A fabricated invoice number or made-up date enters your records as if it were real, and you cannot tell which values are genuine. The fix is one sentence: "If a value is not present, return null and do not guess." This single instruction eliminates most fabrication, a point reinforced throughout Prompting for Data Extraction: A Beginner's Guide.
Mistake 3: Asking the Model to Normalize
Instructing the model to clean and reformat values during extraction invites silent errors.
Why It Costs You
A model asked to convert "$1,200" to a number may misread a European "1.200" or drop a currency symbol, corrupting the value without warning. Extract the raw value as written, then transform it in code where the logic is explicit and testable. Separating extraction from transformation keeps each step verifiable.
Mistake 4: Skipping Examples
Relying on prose descriptions alone leaves the model guessing at your conventions for ambiguous cases.
Why It Costs You
Output becomes inconsistent, handling missing or competing values differently from run to run. A single worked example pairing input with correct output teaches conventions that paragraphs cannot. Include at least one, and make it demonstrate the hard case. The full set of practices is detailed in Prompting for Data Extraction: Best Practices That Actually Work.
Mistake 5: Trusting Output Without Validation
The most expensive mistake is inserting model output straight into a system because it looks like data.
Why It Costs You
Malformed responses, missing required fields, and invented keys slip into your database and surface later as broken reports. Parse and validate every record against your schema in code, rejecting anything that fails. Validation is the safety net no prompt can replace, and it belongs in every pipeline as covered by The Prompting for Data Extraction Checklist for 2026.
Mistake 6: Ignoring Competing Values
When a document contains several candidates for one field, an unguided model picks one at random.
Why It Costs You
A contract with both an effective date and a renewal date may yield either, and you will not know which until a downstream process behaves strangely. Give an explicit disambiguation rule: "If multiple dates appear, return the one labeled as the due date." A clear rule turns a coin flip into a deterministic choice.
Mistake 7: Tuning Only on Clean Documents
Building and testing a prompt against tidy examples produces a prompt that fails on real input.
Why It Costs You
The first irregular document in production, a scanned page or an email with two amounts, breaks a prompt that never saw irregularity. Gather varied samples including the messy ones and test against all of them before shipping. Production input is never as clean as your demo, a reality explored in Prompting for Data Extraction: Real-World Examples and Use Cases.
How These Mistakes Compound
Each of these failures is manageable in isolation, but they rarely occur alone. A pipeline built without a schema tends also to lack validation, because both reflect the same casual mindset of treating extraction as a quick question rather than an engineering task. Understanding how the mistakes reinforce one another helps you see why fixing only the most visible symptom rarely solves the underlying problem.
The Casual-Prompt Trap
Teams that skip the schema almost always skip examples and validation too, because all three feel like overhead when the first test on a clean document looks perfect. The result is a pipeline that appears to work in a demo and fails quietly in production. The cure is not a single fix but a shift in posture: treat the schema, the example, and the validation as the minimum viable contract, not as polish to add later. Adopting that posture early prevents the cluster of mistakes that otherwise arrive together.
Why Silent Errors Are the Real Danger
The unifying thread across these seven mistakes is that every one of them produces output that looks correct. A fabricated invoice number, a mis-normalized currency, a randomly chosen date, and an unvalidated malformed record all enter your system wearing the costume of valid data. This is what makes extraction errors more dangerous than the loud failures of older approaches, where a broken regular expression simply returned nothing. A model returns something, and that something is convincing.
Building Detection Into the Process
Because the errors are silent, the defense has to be active rather than reactive. Sampling records against their sources, tracking validation-failure rates, and reviewing the prompt's handling of absent fields are the practices that surface problems before they accumulate. The structured habits that prevent the whole cluster are collected in Prompting for Data Extraction: Best Practices That Actually Work, and the framework for organizing them into diagnosable stages appears in A Framework for Prompting for Data Extraction.
Frequently Asked Questions
Which of these mistakes causes the most damage?
Trusting output without validation is the most dangerous because it lets every other error reach your system of record undetected. A fabricated value or wrong type that would have been caught by a simple schema check instead becomes a permanent part of your data. Validation in code is the single safeguard that contains the blast radius of all the other mistakes.
How do I know if my prompt is hallucinating values?
Compare extracted records against their source documents for a sample of cases, paying special attention to fields that are sometimes absent. If the model returns a value where the document has none, it is hallucinating. Adding an explicit missing-value rule and re-testing on the same samples will show whether the fix worked. Regular sampling catches drift over time.
Is normalizing values in the prompt ever acceptable?
It can be acceptable for simple, well-bounded cases where the input format is highly consistent, but it carries risk. The safer default is to extract raw values and normalize in code, where the conversion logic is explicit and testable. If you do normalize in the prompt, validate the result carefully and watch for the silent currency or decimal errors that this shortcut tends to introduce.
Why does testing on clean documents fail me later?
Clean documents do not exercise the prompt's handling of missing fields, odd formats, or competing values, which is exactly where real input differs. A prompt tuned only on perfect examples has never been forced to apply its edge-case rules, so the first irregular document exposes gaps you never tested. Gathering messy samples up front makes those gaps visible while they are still cheap to fix.
Key Takeaways
- Define the schema before the prompt so field names and types stay consistent
- Add an explicit missing-value rule to stop the model from fabricating data
- Extract raw values and normalize in code rather than asking the model to reformat
- Include a worked example that demonstrates handling of edge cases
- Validate every record against the schema in code before storing it
- Give disambiguation rules for competing values and test on messy documents, not just clean ones