Skip to content
← All guides

Evaluation & Operations

Flagship field guide

MDX

Prompt Injection and Agent Security

Why untrusted content can redirect agents and how layered controls protect tools, data, and users.

5 min readAgentic Systems Editorial Team

Editorial review: clarity, operational relevance, safety boundaries, and source quality.

Instructions can arrive as data

Agents read webpages, emails, files, and tool results that may contain text designed to manipulate them. The model cannot reliably distinguish every malicious instruction from legitimate content.

Indirect injection arrives inside material the agent is supposed to process: a webpage, document, ticket, email, or tool response. Delimit and label untrusted content, but assume the model may still follow it. Do not let retrieved text define permissions, reveal system data, or authorize a different task.

Limit authority

Use least-privilege credentials, isolate sensitive tools, validate destinations, and require approval for consequential actions. Assume a determined attacker may influence the model's next request.

Containment is the primary defense. Give each task scoped credentials, separate browsing from sensitive actions, allowlist destinations where possible, and require validated user intent at consequential boundaries. Keep secrets out of model context; a prompt cannot reliably prevent disclosure of information the model can see.

Design for containment

Separate trusted instructions from untrusted content, minimize secrets in context, and monitor unusual tool sequences. Security comes from limiting what a compromised decision can do, not from one perfect prompt.

Test end-to-end attacks that combine content manipulation with available tools. Log denied calls, unusual tool sequences, and data movement across trust boundaries. Rotate credentials and invalidate pending work after suspected compromise. Security review must recur when a new tool changes what a successful injection could accomplish.

Practical example

A malicious instruction in a vendor document

A procurement agent reads a PDF containing text that asks it to ignore policy and email contract data elsewhere. The document parser labels all extracted text as untrusted, the research worker has no email tool, and the proposal service accepts only known supplier records. The attempt is logged. Even if the model is influenced, its environment provides no route from document text to sensitive exfiltration.

Field checklist

Apply it in practice

  • Treat external content and tool results as untrusted data.
  • Keep secrets out of model-visible context.
  • Separate read capabilities from consequential tools.
  • Red-team complete trajectories after every authority change.

Decision framework

Questions to answer before you build

Prompt injection cannot be solved by a perfect instruction. Design the system so untrusted content cannot grant authority, expose secrets, or create an uncontrolled path to consequential tools.

Where does untrusted text enter?

Map webpages, files, email, retrieved passages, user content, and tool results. Label them as data and prevent them from redefining policy.

What can a compromised decision reach?

Separate browsing from sensitive actions, scope credentials to the task, validate destinations, and keep secrets outside model-visible context.

How will attacks be detected?

Trace denied calls, cross-boundary data movement, unusual tool sequences, and attempts to override or discover hidden instructions.

Common failure signals

Watch for these warning signs

  • Depending on delimiters or a warning prompt as the primary defense.
  • Giving one agent both broad untrusted-data access and powerful write tools.
  • Testing malicious phrases in isolation rather than complete attack trajectories.

Field manual

Implementation blueprint

  1. 01

    Map trust boundaries

    Label user text, web pages, documents, email, retrieved passages, tool output, memory, and peer-agent messages by origin and authority.

    Deliverable: A data-flow diagram showing where untrusted content meets sensitive tools or secrets.

  2. 02

    Reduce reachable authority

    Remove unnecessary tools, split read and write roles, scope credentials to task and user, and require fresh approval for commitment.

    Deliverable: A capability matrix with no ambient high-impact permission.

  3. 03

    Validate destinations and effects

    Enforce recipients, resource ownership, data classification, amount limits, and allowed state transitions outside the model.

    Deliverable: Deterministic controls at every consequential sink.

  4. 04

    Test complete attacks

    Combine malicious content with realistic tool access, memory, retries, and data-exfiltration goals.

    Deliverable: Adversarial trajectories that measure both unsafe intent and whether controls contain it.

Reusable working artifact

Agent threat-model worksheet

Start with assets and reachable effects, not a list of jailbreak phrases.

ASSETS
Secrets: [credentials, system instructions]
Sensitive data: [customer records, private documents]
Integrity: [records, code, policy, payments]
External effects: [messages, purchases, publications]

UNTRUSTED SOURCES
[user input, retrieved pages, files, email, tool output, memory]

CONSEQUENTIAL SINKS
[write tools, outbound network, file export, privileged retrieval]

ATTACK PATH
source -> model-visible context -> proposed action -> validator -> tool -> effect

CONTROLS BY LAYER
Minimize: remove unused tools and data
Separate: isolate browsing from privileged execution
Authorize: user + task + resource + action + time
Validate: arguments, destination, policy, postcondition
Observe: denied calls, unusual sequences, cross-boundary movement
Recover: revoke, stop, reconcile, notify, preserve evidence

Measurement

Operational scorecard

Excess authorityAvailable tools and permissions not required by the current taskUnused functionality is attack surface, even when never selected in normal tests.
Injection containmentAdversarial runs that reach a prohibited sink or expose protected dataA blocked request still reveals unsafe model behavior; record both attempt and effect.
Approval integrityCommitted actions whose final parameters differ from the reviewed proposalBind approval to exact arguments, user, policy version, and expiry.

Failure drills

Rehearse before the system has real authority

  • Place malicious instructions in a retrieved document that request data from another connected system.
  • Ask an approved read agent to find an indirect route to a write tool and confirm architectural separation holds.
  • Modify action parameters after approval and verify the execution token is rejected.

Selected primary references

Continue with the source material

These sources inform the wider editorial perspective for this topic. They are not presented as line-by-line citations for every statement.

↑ Back to top