This is a checklist you can actually work from, not a poster to admire. It covers what to verify before you ship a system that sends content to a language model, and what to keep verifying once it is live. Each item comes with a one-line reason, because a checklist whose items you do not understand gets skipped under pressure.
Use it as a review gate. If a new feature touches how prompts are built, run the relevant sections before merging. The items are grouped by phase: planning, building, testing, and operating. Skip nothing in the build section; those are the items that prevent silent production failures.
For the reasoning behind these in depth, pair this with the complete guide and the best practices article.
Planning the Budget
Before any code, settle the numbers.
- [ ] Record the exact model and its window in tokens. Windows differ across models and versions; an assumption here poisons everything downstream.
- [ ] Measure the system prompt and tool schemas with the real tokenizer. These are fixed costs charged on every call, and tool schemas are larger than people expect.
- [ ] Reserve maximum output length explicitly. The model cannot borrow input space to finish an answer, so unreserved output gets clipped.
- [ ] Subtract a 10 to 15 percent safety margin. It absorbs tokenization variance so you never run flush against the ceiling.
- [ ] Write down the resulting working budget. This number, not the headline window, governs every later decision.
The step-by-step approach walks through this calculation with worked numbers.
Building Context Assembly
This section is where reliability is won or lost.
- [ ] Assemble every prompt through one function. Scattered string concatenation makes graceful degradation impossible.
- [ ] Assign an explicit priority to each component. When over budget, the system must know what to shed; instructions stay, low-relevance content goes.
- [ ] Place load-bearing content at the start and end. Models attend least reliably to the middle, so critical instructions belong at the edges.
- [ ] Bound conversation history. Without a cap, long chats fill the window and drop the start silently.
- [ ] Summarize older history on a token threshold. Around 60 percent of the working budget, compress old turns and keep recent ones verbatim.
- [ ] Use retrieval for any corpus larger than the budget. Stuffing wastes the window on irrelevant content and dilutes attention.
- [ ] Chunk retrieval content along natural boundaries. Fixed-length splits cut tables and procedures mid-step, causing wrong answers.
These items are the direct countermeasures to the failures in the common mistakes guide.
The Non-Negotiable Guard
- [ ] Count the fully assembled prompt before every send. Design-time estimates drift at runtime; only a measured count is trustworthy.
- [ ] Verify input plus reserved output fits the window. This is the check that catches an oversized prompt before the API does.
- [ ] Shrink by dropping lowest-priority content, never by random truncation. Random cuts remove signal as readily as noise.
If you implement only one item from this entire checklist, make it this guard. It prevents an entire class of incidents on its own.
Testing at the Edges
Average inputs never break. Hard ones do.
- [ ] Test with token-heavy content. Code, JSON, and tables tokenize far heavier than prose and blow past estimates.
- [ ] Test the longest realistic conversation. History growth is the most common path to overflow in chat systems.
- [ ] Test maximum retrieval. Confirm the largest allowed set of chunks still fits with output reserved.
- [ ] Test non-English input if relevant. Other languages tokenize less efficiently and can double counts.
- [ ] Confirm answers stay complete. A clipped answer means output space was not adequately reserved.
The real-world examples show what these edge cases look like in production.
Operating in Production
The job is not done at launch.
- [ ] Log token count per request. Without it, silent truncation and drift are invisible.
- [ ] Break the count down by component where possible. It tells you which part of the prompt is growing.
- [ ] Alert above a usage threshold. Around 80 percent of the window is a reasonable warning line.
- [ ] Watch average request size over time. Conversations lengthen and corpora grow; systems that fit at launch breach later.
- [ ] Review the budget when you change models. A new model's window invalidates every earlier calculation.
A Quick Pre-Ship Pass
When you are about to ship a context-touching change and need a fast gate, verify these five:
- The working budget is documented and current.
- Prompts assemble through one prioritized function.
- A pre-send guard measures and shrinks oversized prompts.
- History is bounded and summarized on a token threshold.
- Token usage is logged with an alert threshold.
If all five pass, you have covered the failures that cause real incidents. The rest of the checklist hardens the edges. For the conceptual model tying these together, see the framework article.
How to Use This Checklist in Review
A checklist only works if it is actually run, so wire it into your process rather than leaving it as a document. Attach the five-item pre-ship pass to any pull request that changes prompt construction, retrieval, or history handling. Make the author confirm each item explicitly in the description, not with a blanket "looks good." The friction is deliberate: the items that get skipped under deadline pressure are exactly the ones, like the pre-send guard, whose absence causes the worst silent failures.
For larger changes, run the full build and testing sections, not just the quick pass. A new retrieval feature, for instance, demands the chunking, maximum-retrieval, and token-heavy items, because those are the failure paths it introduces. Treat the grouping as a menu matched to the change: small prompt tweaks need the pre-ship five, structural changes need the relevant full sections.
Adapting the Checklist Over Time
No checklist is final. As your system matures, fold in items that catch the failures you actually hit. If a non-English input once slipped past, promote that test from optional to required. If a model change once broke the budget, add an explicit gate that blocks model swaps until the budget is recalculated. The goal is a living document that accumulates your team's hard-won lessons, so that each incident leaves behind a check that prevents its recurrence. A checklist that never changes is one that stopped reflecting reality.
Frequently Asked Questions
Which checklist item matters most?
The pre-send guard that counts the assembled prompt and shrinks it if oversized. It is the single item that catches runtime drift and prevents both hard rejections and silent truncation, regardless of how careful your planning was.
Why reserve a safety margin if I have already reserved output space?
Because tokenization varies with content in ways you cannot fully predict at design time. The margin absorbs that variance so an unexpectedly token-heavy input does not push you over the ceiling even after output is reserved. The two reservations protect against different risks.
How often should I revisit the budget?
Any time you change the model or model version, since the window may differ, and any time you notice average request sizes drifting upward in your logs. Treat a model swap as a full recalculation, not a drop-in replacement.
Do I need all the testing items for a simple system?
The token-heavy and longest-conversation tests are essential for almost any system, because they cover the two most common overflow paths. The non-English and maximum-retrieval tests apply only if your system handles those cases, so skip them if they are genuinely irrelevant.
What does logging by component actually buy me?
It tells you which part of the prompt is growing when request sizes drift, so you know whether to tighten history, reduce retrieval, or trim the system prompt. A single total token count tells you there is a problem; a breakdown tells you where it is.
Key Takeaways
- Plan the budget by recording the window, measuring fixed costs, reserving output, and subtracting a safety margin.
- Build prompts through one prioritized assembly function that places load-bearing content at the edges.
- The pre-send guard that measures and shrinks oversized prompts is the single most important item.
- Bound history with token-threshold summarization and use retrieval with natural-boundary chunking for large corpora.
- Test token-heavy content, longest conversations, and maximum retrieval, confirming answers stay complete.
- Log token usage per request, alert above a threshold, watch for drift, and recalculate the budget on any model change.