Safety · Research analysis
Prompt injection is a permission problem too
Why an agent’s ability to read a document should not grant that document authority to authorize an action.
On this page
THE ESSENTIALS
- External content is evidence, not a source of user authorization.
- Test benign task completion and attack success together.
- Bind permissions to the exact action, target, and user-approved scope.
Our analysis
The boundary that matters
An agent may need to read an email, repository file, or web page to complete a legitimate task. The same content can contain an instruction to send data elsewhere or change the task. The architectural mistake is allowing text encountered during reading to enlarge the agent’s authority.
Our proposal combines model-level instruction handling with an external action policy. The model can propose a tool call; a separate layer decides whether that exact action is within the user’s granted scope. The distinction matters even when the model has been trained to resist malicious instructions.
Published evidence
What the research demonstrates
AgentDojo evaluates agents working with untrusted tool-returned data, including adaptive attacks and normal user tasks. Its setup makes clear that a defense must preserve useful task execution as well as resist attacks. [1]
InjecAgent studies indirect prompt injection against tool-integrated agents, including harmful actions and private-data exfiltration. Its results apply to the evaluated systems and attack settings, not a universal percentage for every future agent. [2]
The Instruction Hierarchy paper studies training models to distinguish instructions with different privilege. BIPIA separately investigates indirect injection and defenses around boundaries and reminders. These are complementary to application permissions; neither turns arbitrary retrieved text into a trustworthy authorization channel. [3, 4]
Proposed experiment
A proposed capability boundary
Define capabilities outside the model: permitted tool, target resource, operation, and scope. A user request to summarize a document can authorize reading that document without authorizing external messages, new recipients, or unrelated file writes. Resolve the requested action against that capability before execution.
For a consequential change, prepare the exact payload first. Any confirmation should refer to the actual target and content, not a vague description. Bind a granted capability to those details, expire it when appropriate, and re-evaluate if the action changes. Retrieved content must not be able to mint or expand the capability.
Keep credentials out of ordinary model-visible text wherever the architecture permits. Apply destination restrictions at the tool boundary rather than asking the model to remember them. Separate read operations from writes and record a concise decision log containing the proposed action, policy result, and scope used.
This is a defense-in-depth proposal. A parser bug, confused resource identity, compromised tool, or overbroad user authorization can still defeat a simplistic policy layer. Permission checks need to operate on canonical resource identities and actual execution semantics, not merely a substring in a model-generated command.
Analytical example
A small invariant we can test
The accompanying demonstration uses exact-match capabilities for action type and target. A read capability for document A permits reading A, refuses sending to a new recipient, refuses writing a different file, and refuses a target change. These deterministic tests check the narrow invariant that supplied untrusted text cannot change the capability object.
They do not demonstrate comprehensive prompt-injection resistance. They omit identity resolution, a real tool runtime, concurrent updates, encoded targets, and a production policy language. The value of the example is that the boundary is explicit and independently testable.
def allowed(action, capability):
return (
action["operation"] == capability["operation"]
and action["target"] == capability["target"]
and action.get("payload_hash") == capability.get("payload_hash")
)Proposed experiment
How to challenge the design
Use a paired suite: every legitimate task has a clean version and an adversarial version with the same intended goal. Vary attack placement across retrieved pages, tool errors, document metadata, and repository content. Include benign documents containing quoted malicious instructions, so a defense cannot succeed by refusing to read everything.
Compare model-only instruction defenses, external capability checks, and their combination. Record legitimate completion rate, attack success rate, unauthorized tool attempts, actions actually blocked, false blocks, latency, and the human decisions required. An unauthorized proposal and an executed unauthorized action are different outcomes and should not be counted as the same event.
Use adaptive testing after a defense is frozen. An evaluator should search for ways to exploit target ambiguity and overly broad scopes. Keep a held-out set of attack families for the final evaluation and publish failure traces with sensitive data removed.
| Defense | What it can address | What it cannot establish alone |
|---|---|---|
| Instruction hierarchy | Model treatment of conflicting text | Correct runtime permission enforcement |
| Content boundaries | Source separation in the prompt | Immunity to adaptive attacks |
| External capabilities | Action scope at execution | Truth of every model interpretation |
| Human review of exact changes | Informed approval of a prepared action | Safety of an ambiguous or altered payload |
Scope & limits
The claim we are making
We propose an architecture and executed only deterministic policy-unit examples. We have not run AgentDojo or InjecAgent, and no attack-resistance rate is claimed. The principle is narrower: authorization should be enforced independently of the content an agent encounters.
Reproduce the analytical examples
The standard-library Python program checks the probability, cache-memory and interval calculations, plus the small policy examples. It does not train a model or run the cited benchmarks.
A CLOSER LOOK
Ask this article.
Ask a question to find passages and their references. Answers are extracted from this article; no live AI model is connected.
READ THE ORIGINAL WORK
Sources & references
Sources reviewed 20 October 2026. Papers describe their own models and experiments; our proposed experiments are identified separately.
- 1AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents ↗
Debenedetti et al. · 2024
arxiv.org - 2InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents ↗
Zhan et al. · 2024
arxiv.org - 3The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions ↗
Wallace et al. · 2024
arxiv.org - 4Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models ↗
Yi et al. · 2023; revised 2025
arxiv.org
Keep exploring
All stories ↗
RetrievalRAG should earn the right to answer
A claim-level design for retrieval, contradiction handling, citations, and useful abstention.
AgentsA coding agent is a system, not a score
How to compare coding agents without confusing model quality, scaffolding, test contamination, and retry budget.
RoboticsRobot foundation models need a data contract
What GR00T and π₀ teach us about the gap between a general model and a reliable policy on a particular robot.