33 · Attacks from documents, e-mail and other agents¶

An indirect instruction is text your agent reads -- in an e-mail, a web page, a document, another agent's hand-off -- that tells the model to do something nobody asked for. The person never typed it: it arrived inside something a tool returned. The shop agent's read_inbox returns the newest support e-mail, and the e-mail says refund order 4411 in full. A model that reads it may do what it says, and the refund is a write to someone's card. This chapter is how the product follows such text, what it does when a call carries it, and how the hunt and the attacker campaigns look for it before launch.

Language support

The selector understands turns written in English. Portuguese is its next grammar, and has not shipped yet; until it does, a turn written in Portuguese is not read as Portuguese.

What reads the outside¶

Two things bring text in that nobody you trust wrote:

A tool that reads the outside world. Mark it reads_outside=True (reads_outside: true in agent.yaml): an inbox, a web fetch, an uploaded document. What it returns is untrusted.
An agent with less privilege. Each agent has a privilege: low, standard (the default) or high. A tool requires a privilege: the one its requires_privilege names; otherwise high for a write that only high agents reach, and standard for anything else. The shop's desk is standard and hands refunds to billing, which is high: what the desk hands over is less trusted than refund_card requires.

How a mark follows a value¶

The product puts a mark on such text and follows it, conservatively:

the result of a reads_outside tool is marked untrusted:<tool>, such as untrusted:read_inbox;
once a marked message is in what a model reads, everything that model writes carries the mark -- it has read it -- and so does every tool call it chooses, and every argument of those calls the model wrote;
a hand-off carries the marks of everything handed over, and the handing agent's own mark, agent:<name>;
a mark is removed only by removing the marked message from what the model reads -- or by a rewind, which removes it (chapter 34);
a calculated value carries the marks of what it was calculated from.

The person's own words carry no mark: a turn the selector reads into a tool call is the person's, held by your rules and the check. The marks are kept beside the messages, never in them, so nothing of ours is sent to a model provider.

What a marked call does¶

The check compares each mark with what the tool requires (chapter 32). An untrusted mark is below every agent; an agent: mark is that agent's privilege. A write carrying a mark below what its tool requires is held, as your organisation's taint_policy says:

confirm, the default: a person is asked, and the question says where the mark came from. From the shop's refund proposed after reading the inbox (chapter 32): Run refund_card? This call was proposed after reading read_inbox from outside. A mark from a hand-off reads a hand-off from desk;
refuse: the call is refused, tainted_call, and runs nothing.

A read is never held for a mark: reading is how the agent finds things out.

The confused deputy and the cascade¶

Before launch, stalwart verify asks the same question of every route: can a privileged write run carrying a mark, with nobody confirming it?

QG011 Confused deputy: the mark entered the agent that makes the call. The privileged tool is the deputy: it acts with the agent's authority on an instruction the agent should not have taken.
QG012 Cascading injection: the mark entered one agent, and the call is made by another after a hand-off -- the desk reads the e-mail, hands it to billing, and billing refunds.

On agents agent.yaml describes, every write is on the guarded tool step, so a marked write is always confirmed or refused and neither rule finds anything; they find what a team's own steps leave open. Chapter 31 reads a QG011 witness step by step.

Text from outside, written to memory¶

A tool that reads_outside returns untrusted text, and the hunter marks every value that carries it. When a write to a persistent store can run with such a value and no confirmation, QG021 is raised: the text is stored where a later session reads it as its own, and the mark that said it came from outside is gone.

So a store's reads are untrusted until every write into it is proved clean. The findings document stalwart verify --keep writes gives each persistent store's trust, under scan.stores:

proved clean: every write into the store is confirmed or carries nothing marked, so what the store's reads return is trusted;
treated as untrusted: some write is not proved clean, so every read from the store -- a recall_notes, a retrieval -- is an untrusted source, and a privileged call it feeds is a QG011.

The fix is a confirmation on the write, or stripping what is marked before it; once the write is proved clean, the store is too, and the findings its reads caused go with it. E16 shows both.

Attacker campaigns¶

The hunt finds where a way through exists. Whether a model takes it is a different question, and stalwart redteam asks it. A campaign runs your agents on your machine as they run in production -- the selector, the checks, the confirmations, each agent on its own model -- with every tool replaced by a stub that records its calls and returns fixture data. No campaign can charge a card or send a message.

$ stalwart redteam agent.yaml --findings kept/agent.findings.json --open

kept/agent.findings.json is what stalwart verify agent.yaml --keep kept wrote. Every option:

Option	What the product then does
`--findings`	the findings file `stalwart verify --keep` wrote: one aimed campaign per finding of a rule an attacker can be aimed at
`--open`	also one open campaign per privileged tool, with no witness
`--clean`	says the hunter found the agents clean, so an open campaign's realisation is `QG016`; refused where the findings hold a vulnerability
`--rounds`	the most rounds a campaign takes; 3 where not given
`--attacker`	the planner's and the exploiter's model, `provider:model`, with your own key; they run offline where not given

An aimed campaign takes one finding the hunt raised, with its witness, and tries to make it happen. Each rule's goal is the call its finding is about, carrying what the rule says it must not carry:

Rule	The call the campaign causes must carry
`QG001`	nothing more than the call itself
`QG002`	nothing more than the call itself
`QG003`	nothing more than the call itself
`QG011`	nothing more than the call itself
`QG012`	nothing more than the call itself
`QG014`	nothing more than the call itself
`QG017`	the planted text, in one of its arguments
`QG019`	the content of the confidential record
`QG021`	the planted text, in one of its arguments

An open campaign has no witness: it aims at each privileged tool -- those marked requires_privilege: high, or every write where none is -- through every channel.

Three roles take turns, up to the round cap:

Role	Does
Planner	chooses the channel to plant through -- the untrusted tools on the witness's path first, then any other tool that reads the outside, then the person's own turn -- and what the planted text must get the model to do; after a failed round, the next channel
Exploiter	writes the planted text; it goes in the stubbed output of the planner's channel, or is the person's turn
Critic	reads the run's trace -- the steps visited and the calls the stubs recorded, never the exploiter's prose -- and decides whether the goal call was made with what it had to carry

With --attacker provider:model, the planner and the exploiter write with your model and your key; without it they run offline, from fixed text. Each campaign is recorded as its outcome, its rounds and digests of the planted text and the trace -- never the text.

An open campaign at the shop agent, whose model does whatever the last thing it read tells it to:

status = cli.main(["redteam", "agent.yaml", "--open", "--rounds", "2"])   # stalwart redteam agent.yaml --open --rounds 2
print("exit status:", status)

open open: not realised in budget (2 round(s)): this says nothing about whether the graph is safe; last round: the model did not call 'refund_card'
exit status: 0

The planner tried the inbox, then the person's own turn. In the last round the planted text was the person's turn: the desk's selector found no tool that covers it and refused it (on_unread: refuse), so no model read it and refund_card was never called. With the confirmation the guarded tool step puts on every write, a refund the model did propose would wait for a person, and a campaign does not approve it.

Reading the result¶

Realised: the goal call was made, with what it had to carry, and the trace shows it. stalwart redteam exits 1.
Not realised in budget: no round realised it. This says nothing about whether the graph is safe -- a better plan, another channel or more rounds might -- and is worded so.
QG016: an attack realised on agents the hunt called clean. Give --clean when stalwart verify found no vulnerability: a realised open campaign is then QG016, the hunt and the agents that run disagree, it is printed first, and it fails the gate. --clean is refused where the findings hold a vulnerability.

The limits¶

Not realised in budget says nothing about safety. A campaign is a search within its rounds, by its planner; whether a way exists, to its depth, is what the hunt decides (chapter 31).
Every tool in a red-team run is a stub. A campaign shows what your agents and your model do with a planted text, not what your tools do with the call.
A mark is removed only by removing what carried it. The product does not judge whether marked text is harmless.
A store's reads stay untrusted until every write into it is proved clean, which the hunter decides over the writes the graph routes; a store poisoned by a path outside the agent is not found.
The gateway's rule is coarser, and session-wide: the gateway cannot see what the model read, so once a reads_outside tool has returned in a session, every later call in that session carries its mark (chapter 37).