AGENCYSCRIPT
CoursesEnterpriseBlog
đź‘‘FoundersSign inJoin Waitlist
AGENCYSCRIPT

Governed Certification Framework

The operating system for AI-enabled agency building. Certify judgment under constraint. Standards over scale. Governance over shortcuts.

Stay informed

Governance updates, certification insights, and industry standards.

Products

  • Platform
  • Certification
  • Launch Program
  • Vault
  • The Book

Certification

  • Foundation (AS-F)
  • Operator (AS-O)
  • Architect (AS-A)
  • Principal (AS-P)

Resources

  • Blog
  • Verify Credential
  • Enterprise
  • Partners
  • Pricing

Company

  • About
  • Contact
  • Careers
  • Press
© 2026 Agency Script, Inc.·
Privacy PolicyTerms of ServiceCertification AgreementSecurity

Standards over scale. Judgment over volume. Governance over shortcuts.

On This Page

Treat the Model as Untrusted, AlwaysWhy This Framing WinsWhat It Looks Like in PracticeSeparate Privilege From ExposureThe Two-Path PatternGate the Dangerous ActionsPrefer Constrained Outputs Over Free TextUse Schemas and AllowlistsConstrain Tool Calls TooDetect, But Never Depend on DetectionDetection Is a Tripwire, Not a WallLog Everything ActionableRed-Team Continuously, Not OnceMaintain a Living Attack CorpusRe-Test on Every Model ChangeAccept the Trade-Offs HonestlySpend Where Consequences Are WorstDo Not Trade Away Privilege SeparationMake Defense Part of the Build, Not an AfterthoughtDecide Trust Boundaries During DesignTreat Model Upgrades as Security EventsKeep Humans in the Loop Where It CountsReserve Human Review for High StakesMake the Human Step MeaningfulFrequently Asked QuestionsIf I can only adopt one practice, which should it be?Are detection classifiers worth the added latency and cost?How do I justify the friction of confirmation gates to stakeholders?Does constraining outputs hurt the user experience?Key Takeaways
Home/Blog/Defenses That Survive Contact With Real Attackers
General

Defenses That Survive Contact With Real Attackers

A

Agency Script Editorial

Editorial Team

·December 5, 2023·6 min read
prompt injection defenseprompt injection defense best practicesprompt injection defense guideprompt engineering

There is no shortage of generic advice about prompt injection: separate your inputs, validate your outputs, test your system. All true, all nearly useless without the judgment to apply them well. The practices that actually hold up under a determined attacker come with opinions attached—about what to prioritize, what to skip, and what trade-offs to accept.

This piece offers that opinionated layer. Each practice comes with the reasoning behind it, because a defense you understand you can adapt, while a defense you merely copied breaks the moment your situation differs from the example. These are positions, not platitudes, and a few of them will be uncomfortable.

The through-line is simple: assume injection will sometimes succeed, and design so that success is survivable. Everything below follows from that stance.

Treat the Model as Untrusted, Always

The foundational practice is a mindset shift. Stop thinking of the model as part of your trusted code and start thinking of it as an external service you do not control.

Why This Framing Wins

Once you treat model output as untrusted, the right architecture falls out naturally: you validate everything it produces, you never let it directly trigger consequential actions, and you put checks between it and the rest of your system. Teams that treat the model as trusted internal logic skip all of that and pay for it later.

What It Looks Like in Practice

Wrap the model the way you would wrap a third-party API whose responses might be malformed or hostile. Every output passes through validation. Every action it requests passes through authorization. The model proposes; your trusted code disposes.

Separate Privilege From Exposure

If you take one structural lesson, take this: the model that reads untrusted content should never hold the authority to do irreversible damage.

The Two-Path Pattern

Build a contaminated path that processes untrusted input but can only produce structured, validated results, and a clean path that takes those results and decides on actions using only trusted input. The injection can travel as far as the structured handoff and no further. This single pattern prevents the majority of catastrophic incidents.

Gate the Dangerous Actions

For anything irreversible—payments, deletions, external messages—require a human confirmation or an independent verification step that the contaminated path cannot influence. The friction is worth it precisely where the stakes are highest.

Prefer Constrained Outputs Over Free Text

Open-ended generation is where injection thrives. Narrow the model's room to maneuver.

Use Schemas and Allowlists

When the model's output drives code, require a strict schema or a fixed set of allowed values, and reject anything that does not fit. A hijacked response that tries to smuggle in a new instruction simply fails the shape check. The narrower the valid output space, the less an injection can exploit.

Constrain Tool Calls Too

Validate every tool invocation against an allowlist of permitted actions and argument ranges. This converts "the model can call any tool with any argument" into "the model can request one of a few vetted operations," which is a vastly smaller attack surface.

Detect, But Never Depend on Detection

Classifiers and filters that flag injection attempts are useful. Treating them as your main defense is a trap.

Detection Is a Tripwire, Not a Wall

Use a second-pass classifier to raise alerts and feed monitoring, accepting that motivated attackers will evade it through paraphrasing and encoding. Its job is to make compromises visible and to catch the unsophisticated majority, not to stop everything.

Log Everything Actionable

Record every tool call, argument, and the input that prompted it. When an incident happens, these logs are the difference between a quick root-cause analysis and a week of guessing. They also reveal slow, low-grade probing that no single alert would catch.

Red-Team Continuously, Not Once

A defense you have not attacked is a guess. Make adversarial testing routine.

Maintain a Living Attack Corpus

Keep a growing library of injection techniques—override phrasings, encoded payloads, role-play framings, multi-document attacks—and add every new public method as it appears. Run the whole corpus against your application on every meaningful change.

Re-Test on Every Model Change

This is the practice teams most often neglect. A model upgrade can silently alter behavior and reopen a hole you closed months ago. Gate model version bumps behind your adversarial suite the same way you gate code behind tests.

Accept the Trade-Offs Honestly

Good defense costs latency, money, and friction. Pretending otherwise leads to corners cut in the wrong places.

Spend Where Consequences Are Worst

Concentrate your heaviest controls—confirmation gates, multiple model passes, strict validation—on the highest-consequence paths, and accept lighter protection where the worst outcome is mild. Uniform paranoia wastes budget; targeted paranoia spends it well.

Do Not Trade Away Privilege Separation

You can compromise on detection sophistication or logging depth when resources are tight. The one thing you should not trade away is keeping high-consequence actions separated from untrusted input. That is the control that makes a successful injection survivable.

Make Defense Part of the Build, Not an Afterthought

The teams with the strongest posture do not bolt security on at the end. They weave it into how they design and ship from the first day.

Decide Trust Boundaries During Design

Before you write integration code, draw the line between trusted and untrusted data and decide which components sit on which side. Designing the trust boundary up front is far cheaper than discovering it after an incident and retrofitting separation into a tangled loop. The earlier the boundary exists on paper, the cleaner the eventual architecture.

Treat Model Upgrades as Security Events

A new model version is not a free improvement; it is a behavior change that can reopen closed holes. Put model bumps through the same review and adversarial testing you apply to risky code changes. Teams that treat upgrades casually are the ones surprised when a long-fixed bypass quietly returns.

Keep Humans in the Loop Where It Counts

Full automation is the goal that gets teams into trouble. The mature stance is selective automation with human judgment reserved for the consequential moments.

Reserve Human Review for High Stakes

Automate the low-consequence majority freely, and route only the genuinely risky decisions—large transactions, irreversible deletions, anything touching trust or money—to a person. This keeps the system fast where speed is harmless and careful where mistakes are expensive. The art is drawing that line in the right place, not eliminating humans entirely.

Make the Human Step Meaningful

A confirmation prompt that everyone clicks through without reading is theater, not defense. Give the reviewer the context they need—what action is proposed, why, and what input drove it—so the check is a real decision rather than a reflex. A well-designed human gate catches the injections that slipped past every automated layer.

For the foundations behind these positions, The Complete Guide to Prompt Injection Defense supplies the mechanics, 7 Common Mistakes with Prompt Injection Defense (and How to Avoid Them) shows the anti-patterns, and Case Study: Prompt Injection Defense in Practice walks through these practices applied to a real system.

Frequently Asked Questions

If I can only adopt one practice, which should it be?

Privilege separation. Keeping any model that reads untrusted content away from irreversible actions, behind a confirmation or clean-path gate, contains the worst outcomes and makes every other failure survivable.

Are detection classifiers worth the added latency and cost?

As a tripwire and monitoring signal, usually yes—they catch unsophisticated attacks and make incidents visible. Just do not let their presence justify weakening the architectural controls that do the real protecting.

How do I justify the friction of confirmation gates to stakeholders?

Frame it by consequence. A confirmation step on a payment or deletion costs seconds; an unauthorized transaction or destructive change costs far more. Apply the friction only where the stakes warrant it, and the trade-off sells itself.

Does constraining outputs hurt the user experience?

For conversational replies, keep things natural. Constrain outputs specifically where they drive code or tool calls—the user never sees that layer, and it is where injection does its damage. The two goals do not actually conflict.

Key Takeaways

  • Treat the model as an untrusted external service and the right architecture follows automatically.
  • Separate privilege from exposure with a two-path design so injections cannot reach irreversible actions.
  • Constrain outputs and tool calls with schemas and allowlists to shrink what an injection can exploit.
  • Use detection as a tripwire and logging for visibility, but never depend on detection as your main defense.
  • Red-team continuously, re-test on every model change, and spend your heaviest controls where consequences are worst.

Search Articles

Categories

OperationsSalesDeliveryGovernance

Popular Tags

prompt engineeringai fundamentalsai toolsthe difference between AIMLagency operationsagency growthenterprise sales

Share Article

A

Agency Script Editorial

Editorial Team

The Agency Script editorial team delivers operational insights on AI delivery, certification, and governance for modern agency operators.

Related Articles

General

Prompt Quality Decides Whether AI Earns Its Keep

Prompt quality is the single biggest variable in whether AI delivers real work or expensive noise. The model matters, the platform matters — but the prompt you write determines whether you get a first

A
Agency Script Editorial
June 1, 2026·10 min read
General

Counting the Real Cost of Every Token You Send

Tokens and context windows sit at the intersection of AI capability and operational cost—yet most business cases treat them as technical footnotes. That's a mistake that costs real money. Every time y

A
Agency Script Editorial
June 1, 2026·10 min read
General

Rolling Out AI Hallucinations Across a Team

Most teams discover AI hallucinations the hard way — a confident-sounding wrong answer makes it into a client deliverable, a legal brief, or a published report. The damage isn't just to the output; it

A
Agency Script Editorial
June 1, 2026·11 min read

Ready to certify your AI capability?

Join the professionals building governed, repeatable AI delivery systems.

Explore Certification