Most AI sandboxes die the same death. Someone sets one up for a big experiment, it works, everyone is impressed, and then it sits untouched until the credentials expire and the synthetic data goes stale. Six months later a new project needs a sandbox and the whole thing gets rebuilt from scratch, badly, under deadline pressure.
A playbook fixes that. Instead of treating the sandbox as a one-off, you define the recurring plays, the triggers that kick each one off, the person who owns it, and the order things happen in. The sandbox stops being a project and becomes a capability your team can reach for on demand.
This is that playbook. It assumes you already understand the basics, what an AI sandbox environment is and why isolation matters, and focuses on operating one over time. If you need the foundation first, the complete guide covers it.
The operating principle: plays, not projects
A play is a named, repeatable sequence with a clear trigger and a single owner. The point is that nobody has to invent the process under pressure. When the trigger fires, the owner runs the play, and everyone knows what to expect.
We organize sandbox operations around five core plays:
- Provision — stand up a fresh isolated environment.
- Seed — load masked or synthetic data and baseline configs.
- Run — execute the experiment with full logging.
- Promote — move a validated result toward production.
- Tear down — wipe state and reclaim resources.
Each one has a trigger, an owner, and an exit condition. Let's walk through them.
Play 1: Provision (trigger: a new experiment is approved)
The moment an experiment gets a green light, the provision play runs. The owner, usually whoever requested the experiment, requests an environment from your template. The exit condition is a running, isolated environment with egress controls verified. Don't let anyone start work until egress is confirmed locked down; that single check prevents the most expensive class of accident.
Play 2: Seed (trigger: provision complete)
Provisioning gives you an empty room. Seeding furnishes it. The owner loads the masked dataset, pins the model versions, and sets prompt and config baselines so the run is reproducible. The exit condition is a sandbox that passes a smoke test: a known prompt returns a known shape of output.
Sequencing: the order matters more than the steps
Teams that struggle with sandboxes usually have all the right pieces but run them in the wrong order. The classic mistake is seeding data before isolation is verified, which is how unmasked records end up in an environment that can still reach the internet.
The non-negotiable sequence:
- Isolate first, always. No data enters an environment whose egress isn't locked.
- Mask before you move. Data gets masked at the pipeline, never inside the sandbox.
- Log before you run. Observability must be live before the first prompt.
- Validate before you promote. Promotion is a decision with criteria, not a default.
If this sequencing feels familiar, it's the backbone of a repeatable workflow, which goes deeper on the hand-off mechanics.
Play 3: Run (trigger: smoke test passes)
The run play is where the actual experiment happens. The owner executes the test plan while observability captures every prompt, response, and tool call. For agentic experiments, loop detection and a hard action cap are mandatory. The exit condition is a complete log and a pass or fail against the experiment's success criteria.
Triggers and owners: the part everyone skips
A play without a trigger never starts on time, and a play without an owner never finishes. Write both down explicitly. The most common failure we see is a sandbox with no named owner, so when something breaks, three people assume someone else is handling it and nobody is.
Assigning ownership cleanly
- Experiment owner drives provision, seed, and run.
- Platform owner maintains the templates, masking pipeline, and isolation controls.
- Reviewer signs off on promotion against documented criteria.
Keep these roles distinct even on small teams; one person can wear two hats, but the hats should be named so accountability is clear.
Play 4: Promote (trigger: experiment passes success criteria)
Promotion is the riskiest play because it's where sandbox results meet production reality. The reviewer checks the run against a written checklist, behavior is stable across repeated runs, no unexpected tool calls, data handling clean, cost within budget, and either approves or sends it back. A clear framework keeps these decisions consistent instead of mood-dependent.
Play 5: Tear down (trigger: experiment closed or promoted)
The most neglected play, and the one that keeps your sandbox healthy. Tearing down wipes state, revokes credentials, and reclaims compute. Without it, environments accumulate, costs creep, and stale data lingers as a liability. Make tear down automatic where you can; a sandbox that doesn't reset becomes the very risk it was built to contain. Reviewing real examples and use cases shows how disciplined teams automate this last step.
Running the plays as a cadence, not a one-off
The plays only deliver their full value when they run on a predictable rhythm. A sandbox that fires its plays once and goes quiet provides a fraction of the benefit of one that's exercised weekly. Cadence is what keeps templates current, masking pipelines healthy, and the team fluent in the process.
Building the rhythm
- Run a heartbeat experiment on a schedule even when no project demands it. A trivial weekly run keeps the provision and tear down plays warm and surfaces drift, expired credentials, a renamed template, before a real experiment hits it.
- Review play health at a fixed interval. The platform owner checks that templates still provision cleanly and the masking pipeline still masks. Infrastructure rots silently; a scheduled check catches the rot.
- Track play metrics over time. How long does provision take? How often does promotion bounce back? These numbers tell you where the process is fragile and where it's solid.
A playbook that runs on a cadence becomes muscle memory. When a high-stakes experiment arrives, nobody hesitates, because the team has run the plays a dozen times that month already. That fluency is the real payoff, and it's why mature teams treat the boring heartbeat runs as non-negotiable rather than busywork.
Frequently Asked Questions
How is a playbook different from documentation?
Documentation describes how things work. A playbook prescribes what to do, when, and who does it. Documentation answers "how does the sandbox isolate traffic"; a playbook answers "an experiment was just approved, what happens next and who runs it." You need both, but the playbook is what makes operations repeatable.
Do I need all five plays if I'm a small team?
Yes, though they can be lightweight. Even a solo operator benefits from naming the trigger and exit condition for each play, because it prevents the two killers: starting work before isolation is verified, and never tearing down. The ceremony scales to your size; the discipline shouldn't.
What's the single most important trigger to get right?
Tear down. Provision and run are exciting and tend to happen naturally. Tear down is boring and gets skipped, which is exactly why stale environments and creeping costs are the most common sandbox problems. Automate it if you possibly can.
How do I keep the playbook from going stale itself?
Review it after every promotion. If a play surprised you, an undocumented step, an unclear owner, fix the playbook before you forget. Treat the playbook as a living artifact that the platform owner maintains, not a document written once and abandoned.
Can this playbook handle both model and agent experiments?
Yes, with one addition. Agent experiments require loop detection and a hard action cap in the run play, because agents take actions rather than just generating text. The other four plays stay the same; the run play simply gets stricter guardrails.
Key Takeaways
- Treat the sandbox as a capability run through repeatable plays, not a project rebuilt from scratch each time.
- The five core plays are provision, seed, run, promote, and tear down, each with a trigger, owner, and exit condition.
- Sequence is non-negotiable: isolate first, mask before moving data, log before running, validate before promoting.
- Every play needs a named trigger and a named owner; missing either is the most common cause of failure.
- Tear down is the most neglected and most important play; automate it so stale environments and creeping costs never accumulate.