This is a working checklist, not a reading exercise. Run it against any AI workload before you provision hardware, and you will catch the errors that quietly drain budgets. Every item includes a short justification so you understand why it matters, because a checklist you do not understand is one you will skip under deadline pressure.
It is organized in the order you should actually work through it: define the workload, size memory, optimize, choose hardware, and operate. Skipping ahead — for example, picking a GPU before sizing memory — is how the checklist fails. For the reasoning behind each phase in depth, pair this with our complete guide.
Copy it, adapt it, and make it part of how your team ships AI.
Phase 1: Define the Workload
Get clarity before any numbers.
- [ ] Identify training, fine-tuning, or inference. These have wildly different hardware needs and must not be conflated.
- [ ] Record the model size in billions of parameters. This is the single best predictor of memory needs.
- [ ] Classify as interactive or batch. Interactive cares about latency; batch cares about throughput and cost.
- [ ] State the quality bar. It determines how aggressively you can quantize or shrink the model.
If any item here is unclear, stop and resolve it. Everything downstream depends on these answers, as our step-by-step guide explains.
Phase 2: Size the Memory
Memory is the gate. Get this right before anything else.
- [ ] Calculate base VRAM. Parameters × 2 for FP16 inference, × 0.5 for 4-bit, × 16–20 for full training.
- [ ] Add 25 percent overhead for the KV cache, activations, and framework.
- [ ] Account for context length. Long contexts grow the KV cache and can blow past your estimate.
- [ ] Confirm the model fits your target card with headroom, not at the razor's edge.
A model that "just fits" will fail the moment context grows, so always leave margin.
Phase 3: Optimize Before Provisioning
Optimization changes which hardware you need, so do it now.
- [ ] Apply 8-bit quantization by default. Nearly free in quality, halves memory.
- [ ] Evaluate 4-bit on your real task. Often acceptable and quarters memory.
- [ ] Consider a smaller model. The biggest cost lever of all if quality holds.
- [ ] Plan batching for throughput workloads to keep the GPU busy.
Re-run your memory math after optimizing. Workloads often drop a whole tier here, the move highlighted in our examples.
Phase 4: Choose the Hardware
Now, and only now, pick the GPU and sourcing model.
- [ ] Match GPU tier to worst-case memory, not average.
- [ ] Check memory bandwidth, not just FLOPS, for large-model inference.
- [ ] Estimate sustained utilization honestly.
- [ ] Choose buy, rent, or API — own only above ~50–60 percent utilization.
Be conservative on utilization; most teams overestimate it and overpay, per our common mistakes guide.
Phase 5: Operate Without Waste
Provisioning is not the end. Operations decide your real bill.
- [ ] Set auto-shutdown timers on every rented instance.
- [ ] Use spot or preemptible capacity for interruptible work.
- [ ] Run a daily idle-instance audit.
- [ ] Instrument utilization and throughput dashboards so regressions surface fast.
- [ ] Validate with a small test run before scaling to full production.
These operational habits, drawn from our best practices guide, prevent the slow leaks that dwarf any one-time savings.
Phase 6: Review on a Cadence
A checklist run once and forgotten is worth little. Compute requirements drift, so build in review.
- [ ] Re-run the checklist on every model swap. A new model can change memory needs and tier requirements entirely.
- [ ] Review utilization weekly for live services, monthly at minimum.
- [ ] Revisit buy-versus-rent quarterly as utilization data accumulates and the picture sharpens.
- [ ] Reassess after any major traffic change, up or down, since both can invalidate yesterday's sizing.
The goal is to make this checklist a loop, not a one-time gate. The teams that overspend are rarely the ones who sized wrong at launch — they are the ones who never looked again as conditions changed. This review discipline mirrors the repeatable model in our framework guide.
How to Use This as a Team Tool
A checklist gains power when more than one person uses it the same way.
Make it part of a launch review, where someone other than the implementer walks through each item before a workload goes live. The second set of eyes catches the assumptions the builder made unconsciously — the full-precision default, the optimistic utilization guess, the forgotten KV cache. These are exactly the errors documented in our common mistakes guide, and they are far cheaper to catch in a five-minute review than in a production bill.
Keep the checklist short enough that people actually use it. If it grows unwieldy, prune it back to the items that have actually caught problems for your team. A checklist nobody runs protects nothing, and the discipline of running it matters more than its completeness.
The Three Items That Catch the Most
If you only ever check three things, check these. They catch the overwhelming majority of expensive mistakes.
- Re-run the memory math after optimizing. Sizing against pre-quantization numbers is the most common reason teams over-provision an entire GPU tier.
- Estimate utilization honestly before buying. Optimism here is what turns a sensible-looking hardware purchase into months of paying for idle capacity.
- Set an auto-shutdown timer on every rented instance. Idle GPUs are the single largest source of wasted spend, and a timer eliminates them for free.
The rest of the checklist is valuable, but these three sit at the intersection of high cost and easy oversight. A team that reliably does only these will already spend far less than one that ticks every other box but skips them. The full sequence above adds rigor; these three add the most savings per minute of effort, and they connect directly to the failures in our common mistakes breakdown.
Frequently Asked Questions
What is the most-skipped item on this checklist?
Re-running the memory math after optimizing. Teams optimize but size hardware against the pre-optimization numbers, so they over-provision. Always recalculate VRAM once quantization and model choice are settled.
Why size for worst case instead of average memory?
Because running out of VRAM causes hard failures, not graceful slowdowns. Average-case sizing works until a long context or a traffic spike pushes you over, at which point the workload simply crashes.
How often should I run the idle-instance audit?
Daily for rented capacity. Idle GPUs bill continuously and silently, and instances left running over a weekend can quietly erase a month of savings. A quick daily check pays for itself many times over.
Do I need dashboards for a small project?
Even basic utilization and throughput visibility helps. You cannot manage what you cannot see, and the cheapest dashboards still catch idle waste and creeping overspend before they compound.
Can I use this checklist for fine-tuning?
Yes. The structure is identical; only the memory multiplier in Phase 2 changes to the training range. Fine-tuning sits closer to training than inference in its hardware demands.
Key Takeaways
- Work the checklist in order: define, size memory, optimize, choose hardware, operate.
- Memory is the gate — size it before touching any other decision.
- Always re-run the memory math after quantizing; optimization changes the answer.
- Match GPU tier to worst-case memory and check bandwidth, not just FLOPS.
- Own hardware only above ~50–60 percent honest sustained utilization.
- Operational habits — shutdown timers, spot capacity, daily audits — decide your real bill.