The most dangerous sandbox is not the one that obviously fails. It is the one that appears to work. A broken sandbox that throws errors gets fixed. A sandbox with a quiet hole in its wall keeps running, builds false confidence, and leaks on the worst possible day.
This article catalogs the seven failures we see most often. For each, you will get the underlying cause, the cost when it goes wrong, and the specific practice that prevents it. These are not hypotheticals. They are the patterns that turn a safety system into a liability.
If you are setting up a sandbox for the first time, read this alongside the step-by-step approach. Knowing the failure modes in advance changes how you build.
Mistake 1: Locking the data but leaving the network open
This is the single most common error. Teams carefully provision synthetic data, feel safe, and never restrict outbound network access. The agent then reaches out to the internet, or worse, to internal hostnames, and the isolation is decorative.
Why it happens: Network policy is invisible. Nobody sees the open door until something walks through it.
The cost: Data exfiltration, calls to internal services, or an agent pulling in a malicious dependency.
The fix: Deny outbound network by default and allowlist only the specific endpoints you need. Block your cloud metadata endpoint explicitly.
Mistake 2: Putting real data in "just this once"
Someone decides synthetic data is too unrealistic and quietly loads a production export to get a better test. Now the sandbox holds exactly what it was built to keep out.
Why it happens: Synthetic data can feel too clean, and the temptation to use the real thing is strong under deadline pressure.
The cost: A leak from the sandbox is now a real breach, not a harmless test.
The fix: If you need realism, mask production data so every sensitive value is replaced before it enters the box. Make masked data so easy to generate that nobody reaches for the raw export.
Mistake 3: Reusing stale environments
A clean sandbox takes effort to create, so people reuse yesterday's. State from a previous run, leftover files, cached credentials, modified configs, contaminates today's experiment.
Why it happens: Provisioning is slow or manual, so disposability gets abandoned in practice even when it exists in principle.
The cost: Irreproducible results and, occasionally, leftover credentials granting access nobody intended.
The fix
- Script provisioning so a fresh sandbox is faster than cleaning an old one.
- Destroy and recreate between meaningful runs, on principle.
- Treat any long-lived sandbox as suspect by default.
Mistake 4: Granting the agent every capability
To avoid fiddling with permissions, teams hand the agent broad access: file system, network, shell, email, the works. Each capability is a fresh escape route for a mistake.
Why it happens: Least privilege is more configuration upfront, and broad access "just works" in the moment.
The cost: A confused agent does something with reach, sends a message, makes a purchase, modifies a system, because it was allowed to.
The fix: Grant the minimum tools the task requires and nothing more. The reasoning behind least-privilege design is covered in depth in our best practices guide.
Mistake 5: Running without observability
The agent runs, something odd happens, and there is no log to explain it. The team is left guessing, and the same mystery recurs.
Why it happens: Logging feels like overhead until the first incident you cannot diagnose.
The cost: Unreproducible bugs, no audit trail, and no way to prove the sandbox behaved correctly.
The fix: Log every prompt, tool call, command, and output from the very first run. Observability is not a debugging luxury; it is the evidence layer of the whole system.
Mistake 6: No spend or rate limits
An autonomous agent gets stuck in a retry loop overnight and burns a startling token bill, or hammers an API until it gets rate-limited or billed into the next tier.
Why it happens: Caps feel unnecessary until an agent loops, and agents loop more often than people expect.
The cost: A surprise expense that turns an experiment into a budget conversation.
The fix: Set a hard token spend cap and an action rate limit as a circuit breaker before any unattended run. Treat these as non-negotiable defaults.
Mistake 7: Never testing the walls
The sandbox is built, assumed correct, and never challenged. Nobody actually checks whether the isolation holds, so a hole sits undiscovered until production.
Why it happens: Once something appears to work, testing it feels redundant.
The cost: The most expensive failure of all, discovering the wall was broken only after data has already escaped.
The fix: Test containment adversarially. Instruct an agent inside to reach a forbidden endpoint, access a production hostname, or persist a file past teardown. Run these checks on every config change and on a schedule. The full procedure is in our step-by-step guide, and the broader context in the complete guide.
The pattern behind all seven
Step back from the individual failures and a single shape emerges: every one of them is a case of partial safety masquerading as complete safety. The team did something real, locked the data, built the walls, ran the agent, and then assumed the part they did stood in for the whole.
That is what makes these mistakes so durable. They are not the result of laziness or ignorance. They are the result of stopping one step short, and the missing step is invisible precisely because the visible steps look like progress. A locked data layer feels like security. A running agent feels like a working system. The gap is silent until it is not.
The corrective mindset is to treat isolation as a property of the weakest layer, never the strongest. Your sandbox is exactly as safe as its most open dimension, the same way a chain is as strong as its weakest link. When you evaluate your own setup, do not ask "what did I protect?" Ask "what did I leave open?" The second question finds the holes the first one hides. Our best practices guide reframes every control around this single idea.
How to audit yourself against this list
A practical way to use these seven is as a recurring self-audit rather than a one-time read.
- Schedule it. Once a month, walk your live sandbox configuration against all seven failures explicitly. Drift is silent, so the calendar has to be the trigger.
- Assign a skeptic. Have someone who did not build the sandbox try to find the open dimension. Builders are blind to their own assumptions; a fresh skeptic is not.
- Treat near-misses as findings. Any time the sandbox catches something, log it and ask which of these seven it relates to. A caught failure today is a roadmap to the gap you have not found yet.
The teams that avoid these mistakes are not smarter. They simply assume their sandbox is broken until a recent adversarial test proves otherwise, and they re-prove it on a schedule.
Frequently Asked Questions
Which of these mistakes is the most dangerous?
The open-network mistake, because it pairs an invisible failure with a high-impact outcome. Teams feel safe from carefully locking the data layer while leaving the most direct exfiltration path wide open. It is the failure most likely to be both present and unnoticed.
How do I catch a stale-environment problem before it bites?
Make freshly provisioned sandboxes faster to create than dirty ones are to clean. When the clean path is the fast path, reuse stops happening on its own. If you find yourself manually deleting files to "reset" a sandbox, your provisioning is too slow.
Is least privilege worth the extra setup for a small experiment?
Yes, because the experiments most likely to surprise you are the small, casual ones nobody watched closely. Least privilege caps the damage of exactly those unsupervised runs. The setup cost is one-time; the protection is every run.
My sandbox has worked fine for months. Do I still need adversarial tests?
Especially then. Isolation erodes quietly as people add allowlist entries and permissions for convenience. A sandbox that worked at setup can develop a hole months later without any error to signal it. Scheduled checks catch that drift.
Can a single mistake from this list cancel out everything else I did right?
Often, yes. Isolation is only as strong as its weakest layer. Perfect data masking means little if the network is open, and tight network rules mean little if real data is inside. The layers reinforce each other, so one gap can undo the rest.
Key Takeaways
- The most dangerous sandbox is one that appears to work while hiding a quiet hole.
- Open networking is the most common and most costly failure; deny outbound by default and allowlist deliberately.
- Disposability fails in practice when provisioning is slow, so make fresh environments faster to create than stale ones are to clean.
- Least privilege, full observability, and spend caps are not optional extras; each one prevents a specific, recurring failure.
- Test your walls adversarially on a schedule, because isolation erodes quietly and a single gap can undo everything else.