The Stalwart user guide¶

This guide is for the people who build agents with Stalwart, and for the people who answer for those agents. It says what each product does, how to use it, and where each guarantee stops.

The products¶

One API key reaches all of them. The guide names them as the website and the API reference do.

• The Bot Builder builds your own AI agent from the tools your business already has, declared in one file, with your rules checked on every tool call before it runs. A component inside it, the selector, reads each turn of a conversation and picks the tool by a named rule, with no model call.
• Agent Graph Verification checks every route through an agent, or a team of agents, before it goes live, and returns each route that reaches a tool without its check, its confirmation or its order rule.
• The Guardrail Bug Hunter finds every way through your guardrails to a forbidden action, shows each one happening, and hands you a fix it has already checked by scanning again.
• Guardrail Verification answers whether one action can fire without its safety context, or a whole table of such questions, and returns a counterexample with a repair, or a certificate.
• Guardrail Writing drafts a guardrail policy from a rule written in prose, on your own model, then verifies the draft and repairs it where it is wrong.
• Random Forest Verification checks a trained random forest against robustness, monotonicity, fairness and reachability properties, or searches around real records for the smallest change that flips a decision.

The hunter and the selector, together¶

The two are built to be used together. A guardrail can make every tool action wait for the selector's decision: the selector reads the turn, and the guardrail runs a tool only where the selector's plan says run for that tool. The Guardrail Bug Hunter then searches every path through the guardrail for one that reaches a tool without that decision, and fails the pull request that adds one. Chapter 29 works through that example. A clean scan proves the policy sends every tool through the selector; it does not prove the selector chose the right tool, which chapter 20 explains.

One agent, in one file¶

A team puts the products to work through one agent, without learning an agent framework. You declare the agent in one file: its tools, its rules, and which writes undo which. Before it runs, its routes are verified; while it runs, every turn is read and every tool call is checked on your own machine; and a run that fails is undone, last write first. This is the shop agent the guide uses throughout Part I:

# The shop's support desk, and the billing desk it hands refunds to.
agents:
  desk:
    model: "anthropic:claude-sonnet-5-5"   # your provider; the key comes from your environment
    tools: [get_order, charge_card, send_receipt, read_inbox]
    hands_off_to: [billing]
    on_unread: refuse                      # a turn no tool covers is refused, by name
  billing:
    privilege: high
    tools: [refund_card]
tools:
  - python: shop.tools                     # every public function of shop/tools.py
rulebook: ./policy.co                      # the rules every call is checked against
runs: atomic                               # a failed run is undone, last write first
storage: sqlite:///runs.db                 # where runs and waiting confirmations are kept

Part I builds it step by step, in the words of the file above and in no framework's.

Where to start¶

• If you only read one page: the whole guide on a page, with its limits.
• Quickstart: an agent declared, verified and served; a turn read; a policy scanned; a verdict and a certificate. About fifteen minutes.
• What this service does, and what it does not claim: chapter 1. Read its second half before you quote a result to anyone.

The guide is in five parts: the Bot Builder, the Bot Builder's selector, the Guardrail Bug Hunter, the hunter and the selector together, and verification and writing. Chapter 39 is for governance and security teams: what each product maps to in the OWASP Top 10 lists of 2026 and the NIST AI Risk Management Framework, and what such a mapping is not. The appendix holds the reference tables, the glossary and the changelog.

Checking our work without us¶

A certificate can be checked with no key, no network and none of our software. The certificate specification says how to read one, the reference checker is one file you can read in an afternoon, and the certificate corpus gives you real certificates to run it on.

The guide offline. The whole guide in one file, for reading with no network or for an audit file: English · Portuguese.

Each file's SHA-256 is in SHA256SUMS, so a copy can be checked against the one published here. Links that leave the guide -- the API reference, the website -- need the network, and say so; the offline guide has no search.

Elsewhere¶

• stalwart.it: the products and the plans, and where you sign up.
• The API reference: every operation, which you can try with your own key.
• The package's documentation: stalwart-agents, page by page, and its Python reference.

Chapter numbers are identifiers: a chapter keeps its number whatever page it moves to, so an audit file that cites chapter 7 keeps pointing at the same chapter.

If you only read one page¶

This page is the guide in brief, limits included. Every statement on it is explained, with its evidence, in the chapter it links.

What each product does¶

• The Bot Builder builds your own agent from the tools you already have: Python functions, MCP servers, OpenAPI operations, LangChain tools or a TypeScript project. You declare the agent in one agent.yaml file or in Python (chapter 30).
• Agent Graph Verification checks every route through an agent before it runs (chapter 47, and chapter 31 for an agent you built with the Bot Builder).
• The Guardrail Bug Hunter scans a guardrail policy, a router or an agent graph for every way through it to a forbidden action (chapter 28).
• Guardrail Verification decides whether an action can fire without its safety context (chapters 3 to 12).
• Guardrail Writing drafts a policy from a rule in prose, on your own model, and verifies the draft (chapter 13).
• Random Forest Verification checks a trained forest against the properties you state (chapter 14).

The two that are used together most are the selector, which is part of the Bot Builder, and the bug hunter: a guardrail makes every tool wait for the selector's decision, and the hunter looks for every path that skips it, failing the change that adds one (chapter 29).

How an agent is kept in line¶

• Before it runs, its routes are verified: every path to a write without its check, its confirmation or its order rule is a finding, and an agent with such a path does not start (chapter 45). stalwart verify makes the same check a gate in CI (chapter 46).
• On every turn, the selector reads the turn and decides the tool, a question to ask, or a refusal, by a named rule. A model is called only where your settings send a turn to one (chapter 20).
• On every tool call, whoever chose it, the call is checked against your catalogue, your rules and the session's history. The answer is allow, confirm (a person approves the exact arguments) or refuse (chapter 32).
• When a run fails, its writes are undone, last first, each undo checked like any call (chapter 34).
• Before a reply is shown, it is held to what was actually done, and checked for personal data and wording you forbid (chapters 42 and 43).
• All of that runs on your own machine, in the licensed runtime. Your agents' calls and your documents never reach our service, and a user's turn reaches it only as the words of a turn somebody rated with a thumb (chapter 25). Otherwise only counts do (chapter 38).

What it does not claim¶

These are the limits a reader most needs, in short. Each chapter states its own under a heading of its own.

1. A verification proves the question you asked, and no other. One run proves one proposition. A policy is only as verified as your list of questions is complete (chapter 1).
2. A certificate establishes a derivation, not that its premises are your policy. Checking a certificate shows the equations it starts from have no common solution; it does not show those equations are a faithful compilation of the policy (chapter 7).
3. A verdict over a policy the compiler did not fully read is not a verdict. A line the parser skipped is a missing edge, and a missing edge reads as a bypass or a pass that means nothing (chapter 6).
4. A patch is proved to close the hole, and only sometimes proved not to over-block. The second proof has its own outcome, and only proved_free makes that claim (chapter 8).
5. A stall on the guardrail side is acted on as a bypass; on the random forest side it concludes nothing (chapters 6 and 14).
6. A clean agent scan proves no path reaches a tool without its check, to the depth checked. It does not prove the model chooses well, and it reads your own functions as able to do anything (chapter 31).
7. An attack not realised says nothing about safety (chapter 33).
8. A rollback does not undo an irreversible step, and a saga compensates rather than locks (chapter 34).
9. The reply checks remove what the system's facts do not support; they do not prove a reply true (chapter 42).
10. An OWASP or NIST mapping is not a certification, and the hunter's OWASP coverage is at the policy level (chapter 39).

How to check our work¶

A certificate can be checked with no key, no network and none of our software: read the specification, or run the reference checker, one short file under the MIT licence, over the certificates in the corpus.

Your account¶

Sign up at stalwart.it with Google or an emailed link. The first key is shown once on the welcome page; every key after that is yours to mint in the console (chapter 5). What each plan includes is in chapter 40; what is kept, for how long, and what is never kept, is in chapter 16.

Quickstart¶

This chapter is being written. Until it is published, the Python SDK reference and the HTTP API reference describe every class, method and operation it will cover.

30 · Your first agent¶

The Bot Builder is the stalwart-agents package. You declare an agent in one file, or in a few lines of Python: its tools, the words that say what each tool does, its rules, and which agent it may hand a conversation to. You check it, and you run it. You do not need to learn an agent framework for any of this, and this part of the guide never asks you to.

Before you start¶

You need an account and its key (chapter 5), and the licence activated once on each machine that runs your agents:

$ pip install "stalwart-agents[anthropic]"    # or openai, google, mistral, ollama; or none, for no model
$ export QLS_API_KEY=qls_live_...
$ stalwart licence activate

Nothing in the package works without the key and an activated licence, except activating it, stalwart doctor, which says what is missing, and stalwart check, which checks a certificate. What a licence is, and what works offline, is chapter 38.

Where the runtime is installed, a turn and a tool call need no network: the runtime answers both on the machine that runs your agent.

The shop agent, in one file¶

This guide follows one agent through Part I: a small shop's support desk. It reads orders, charges cards, e-mails receipts and reads the support inbox, and it hands refunds to a billing desk that has more privilege:

# The shop's support desk, and the billing desk it hands refunds to.
agents:
  desk:
    model: "anthropic:claude-sonnet-5-5"   # your provider; the key comes from your environment
    tools: [get_order, charge_card, send_receipt, read_inbox]
    hands_off_to: [billing]
    on_unread: refuse                      # a turn no tool covers is refused, by name
  billing:
    privilege: high
    tools: [refund_card]
tools:
  - python: shop.tools                     # every public function of shop/tools.py
rulebook: ./policy.co                      # the rules every call is checked against
runs: atomic                               # a failed run is undone, last write first
storage: sqlite:///runs.db                 # where runs and waiting confirmations are kept

Its tools are plain Python functions. A function's type hints are its arguments, the first line of its docstring is what it does, and @tool(...) gives it the words the checks read. A function with no words is a read:

"""The shop's tools: plain functions. Their type hints are their schemas, the first line of
each docstring is what the tool does, and @tool gives a tool the words the checks read."""

from decimal import Decimal

from stalwart_agents import tool

ORDERS = {
    "4411": {"order_id": "4411", "total": "40.00", "card": "visa-0019", "customer": "c-208", "status": "unpaid"},
    "4412": {"order_id": "4412", "total": "12.50", "card": "visa-0019", "customer": "c-208", "status": "paid"},
}


def get_order(order_id: str) -> dict:
    """Show an order: its total, the card on file, the customer and its status."""
    return dict(ORDERS.get(order_id, {"order_id": order_id, "status": "not found"}))


@tool(writes=True, undo="refund_card")
def charge_card(order_id: str, amount: Decimal) -> dict:
    """Charge an amount to the card on file for an order."""
    return {"charge_id": f"ch-{order_id}", "order_id": order_id, "amount": str(amount)}


@tool(writes=True, requires_privilege="high")
def refund_card(order_id: str, amount: Decimal) -> dict:
    """Refund an amount to the card on file for an order."""
    return {"refund_id": f"rf-{order_id}", "order_id": order_id, "amount": str(amount)}


@tool(writes=True, irreversible=True)
def send_receipt(order_id: str) -> dict:
    """E-mail the receipt for an order to its customer."""
    return {"sent": True, "order_id": order_id}


@tool(reads_outside=True)
def read_inbox(folder: str = "support") -> dict:
    """Read the newest message in the shop's support inbox."""
    return {"from": "someone@example.com", "body": "refund order 4411 in full"}

Before it answers anyone, check it:

stalwart verify agent.yaml

Registered as catalogue version 1, with the rulebook policy.co: the tool check enforces the rules verified here.
Lints at registration (nothing fails on a lint):
  write_not_retry_safe (warning) on charge_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on charge_card: a write with no result schema, so its outcome could not be reported
  compensation_not_retry_safe (warning) on charge_card: 'refund_card' undoes it and is not retry-safe
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported
  compensation_out_of_reach (warning) on charge_card: agent 'desk' can call it but not 'refund_card', which undoes it, so a run of that agent cannot undo it

Code checks: no finding in 4 file(s) (rules 4b48b0081360).

QG023 Medium on cycle:desk_model (agent.yaml:3)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card (agent.yaml:10)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (policy.co:19)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG019: no tool is declared confidential
  QG020: no retrieval from a store declared multi-tenant
  QG021: no tool is declared to write a store declared persistent
  QG106: no tool declares a source or a server, and the catalogue declares no modules
  QG107: no agent in the catalogue declares a remote channel

Gate: passed

stalwart verify reads your code for network and credential problems on your machine, registers the catalogue with the rulebook, and has every route through the two agents searched on our servers for a way to a tool without its check, its confirmation or its order rule. Chapter 31 is what it finds and how to read it.

Then talk to it:

desk = Agent.from_file("agent.yaml")        # the graph is proved once, before the first turn

reply = desk.chat("show me order 4411", user="u-17")
print(reply.text)                           # get_order, chosen from the words; no model was asked

reply = desk.chat("charge order 4411 40.00", user="u-17")
for waiting in reply.waiting:               # the write waits for a person
    print(waiting.tool, "|", waiting.wording, "|", waiting.decisions)
    for argument in waiting.arguments:      # each argument, and where its value came from
        print("  ", argument)

reply = desk.confirm("u-17", "approve")     # runs now, with exactly the arguments shown
print(reply.text)

{"card": "visa-0019", "customer": "c-208", "order_id": "4411", "status": "unpaid", "total": "40.00"}
charge_card | Run charge_card? | ['approve', 'skip', 'cancel_all']
   {'name': 'amount', 'value': 40, 'origin': 'typed'}
   {'name': 'order_id', 'value': '4411', 'origin': 'typed'}
{"amount": "40", "charge_id": "ch-4411", "order_id": "4411"}

The read was answered with no model call: the selector read show me order 4411 against what each tool says, and chose get_order. charge order 4411 40.00 names the charge, the order and the amount, in the order a person types them. The charge waited for a person, showing each argument and where its value came from, and ran only once approved, with exactly those arguments (chapter 32).

To serve it over HTTP, with a chat interface at / that shows confirmations, refusals and rollbacks:

$ stalwart serve agent.yaml

Chapter 35 is the interface and the routes behind it.

The same agent in Python¶

Everything the file says can be said in Python, and means the same:

from stalwart_agents import Agent

from shop.tools import charge_card, get_order, read_inbox, refund_card, send_receipt

billing = Agent("billing", tools=[refund_card], privilege="high", rulebook="policy.co")
desk = Agent("desk", tools=[get_order, charge_card, send_receipt, read_inbox], hands_off_to=[billing],
             model="anthropic:claude-sonnet-5-5", on_unread="refuse", rulebook="policy.co",
             storage="sqlite:///runs.db")

from_file = Agent.from_file("agent.yaml")
print(desk.catalogue_digest() == from_file.catalogue_digest())   # the same catalogue, either way

True

The file and the Python agent give the same catalogue, with the same digest, so a team can start in either and move to the other.

What agent.yaml says¶

Every word the file takes, generated from the package:

A word the file does not know is an error at its line, never ignored (below).

Where tools come from¶

• python: shop.tools takes every public function of the module, imported from the folder the file is in. Its schema comes from its type hints.
• mcp: https://tools.internal/mcp takes the tools an MCP server lists, with their schemas and the hints the server gives: a tool the server marks read-only reads, one it marks destructive writes, and one it says nothing about writes. tools: [...] keeps only those named; version: or digest: pins the server. Each user of your agent gets an MCP session of their own on the server, so what a server keeps for one user never reaches another.
• openapi: ./crm.yaml makes each operation of an OpenAPI document a tool: GET and HEAD read, any other method writes unless the operation says otherwise in x-stalwart.
• name: refund_card, with words beside it, adds words to a tool from any source.

Chapter 21 has the other ways in, such as the exported functions of a TypeScript project.

A tool's words¶

The words are what turns a function into a tool the product can govern. Each sets one field of the catalogue the checks read:

You do not need most of them to start. A read needs none; a write needs writes=True; a write that can be undone needs undo=; one that cannot needs irreversible=True. The rest make the checks more precise, and the bug hunter tells you when one is missing (chapter 31).

Agents, privilege and hand-offs¶

An agent may hand a conversation to another it names in hands_off_to. Each agent has its own tools, and calls only those. privilege says how much a hand-off from an agent is trusted: an instruction that entered a low-privilege agent, from an e-mail say, is marked when it reaches a high-privilege one, and a write it leads to is confirmed by a person (chapter 33).

The model¶

model: "anthropic:claude-sonnet-5-5" names a provider and a model; the key is read from your environment, under the provider's usual variable, and stays yours. A missing key is reported before anything is sent, naming the variable. A model can also be a plain Python function that takes the conversation and returns the reply. We hold no model credential, anywhere.

The model is called only where your settings send a turn to it. Every turn is read by the selector first, and a turn that names a tool, needs a question asked, or is declined never reaches the model. A turn nothing reads is asked to be said again once; then on_unread decides: refuse, model, ask (the person says whether the model may have it) or answer, from your documents (chapter 44). With no model, on_unread: refuse, no hands_off_to, replies from your phrasings and answers in your documents' own sentences, no model is ever called (chapter 45).

Runs and storage¶

runs: atomic, the default, undoes a failed run, last write first (chapter 34). storage: is where runs and waiting confirmations are kept, so that a confirmation can be answered after a restart, or on another worker: sqlite:///runs.db for one machine, postgres://... for several.

The commands¶

Generated from the stalwart command itself:

The errors¶

Every error the package raises names what to change, in the words of agent.yaml, never in the terms of the libraries underneath:

A file error names its line:

agent.yaml:7: agent 'desk' lists 'refund', which no tools source declares

The limits¶

What stalwart-agents decides, and what stays yours:

• It decides which tool a turn names, whether a call may run, waits for a person or is refused, what is undone when a run fails, and what a reply may say about what was done.
• Your functions decide what they do. A tool's words are taken as you state them: a tool declared a read that writes is governed as a read. The code checks read your code for network and credential problems, and nothing else about what it does (chapter 46).
• Your model decides what it says and which tool it asks for, where your settings let it. Every call it asks for is checked; its choices are not proved good (chapter 31).
• A tool reachable outside the agent can be called around it. The checks govern the calls the agent makes; chapter 37 puts the same check in front of your tools for everything else.

31 · Checking an agent before it runs¶

stalwart verify agent.yaml checks an agent before it serves anyone. It reads your code on your machine, registers the agent's catalogue with its rulebook, and has every route through your agents searched on our servers for a way to a tool without its check, its confirmation or the step it must follow. It prints each finding with its rule, the line of agent.yaml it comes from, its OWASP entries and its fix, and exits non-zero when the gate fails, so it stands in CI as it stands on your machine.

What it does, in order¶

1. The code checks, here. Every Python file of the folder agent.yaml is in is read on your machine for network and credential problems, with the tools the file names taken as the functions whose parameters are tool arguments. Any finding fails the gate. Only its counts by rule leave your machine, with the registration (chapter 46).
2. The registration. The catalogue -- every tool with its words -- is registered as a new version, with the file's rulebook: compiled into the rules the tool check enforces. So the rules the agent is checked against here are the rules it runs under. Registration gives lints; nothing fails on a lint.
3. The hunt, on our servers. The agents' graph, the catalogue and the rulebook are scanned by the bug hunter as one hunt job, under its gate. The search runs on Stalwart's servers, never on your machine. A hunt that could not run is an error, never a clean result.
4. The report. The findings are printed one per block, the most severe first.

The shop agent of chapter 30, checked:

stalwart verify agent.yaml

Registered as catalogue version 1, with the rulebook policy.co: the tool check enforces the rules verified here.
Lints at registration (nothing fails on a lint):
  write_not_retry_safe (warning) on charge_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on charge_card: a write with no result schema, so its outcome could not be reported
  compensation_not_retry_safe (warning) on charge_card: 'refund_card' undoes it and is not retry-safe
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported
  compensation_out_of_reach (warning) on charge_card: agent 'desk' can call it but not 'refund_card', which undoes it, so a run of that agent cannot undo it

Code checks: no finding in 4 file(s) (rules 4b48b0081360).

QG023 Medium on cycle:desk_model (agent.yaml:3)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card (agent.yaml:10)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (policy.co:19)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG019: no tool is declared confidential
  QG020: no retrieval from a store declared multi-tenant
  QG021: no tool is declared to write a store declared persistent
  QG106: no tool declares a source or a server, and the catalogue declares no modules
  QG107: no agent in the catalogue declares a remote channel

Gate: passed

Read it from the top:

• Registered as catalogue version 1, with the rulebook policy.co: the version the tool check now enforces. With no key, the line says the catalogue was not registered, and the lints come from the runtime as registration would give them.
• Lints at registration: what the catalogue leaves unsaid, such as a write with no undo. Each names the tool and what to add; none fails anything.
• Code checks: the files read and the version of the rules they were read with.
• Each finding: the rule, its severity and the tool, with the file and line that declared it; what was found, in one sentence; its OWASP entries; and the fix. Here QG023 says the desk's loop has no budget but the run's step limit, and QG105 says refund_card is a write with no undo, on a route with another write.
• Rules with nothing to read: rules that raised nothing because nothing they read was declared (below).
• Gate: passed or failed. A finding under a rule that gates fails it, and so does any code-check finding. Each rule's page says whether it gates by default; QG023 and QG105 do not, so the shop agent passes.

The command's exit status, generated from the command itself:

--keep DIR writes what was checked into a folder: agent.graph.json, the agents' graph as the hunter read it; agent.catalogue.json, the catalogue; and agent.findings.json, the findings document. Every option:

The questions it answers for every tool¶

For every tool, on every route through your agents, to the depth stated, the hunt asks:

• Can it run without its check, or can the check be got past? Over several turns, can an approval given at one turn be spent at another?
• Can a write run carrying a mark from outside -- an e-mail, a web page, another agent with less privilege -- with nobody confirming it?
• Can an irreversible step come before steps that will be undone if the run fails, without its confirmation saying so?
• Can a tool run out of its declared order?
• Can a route go anywhere, because nothing declares where it may go?
• Can a write fail to be undone, because nothing undoes it and nothing says it cannot be?

Each question is a rule, and each rule has its page:

Why each matters, and how to fix it:

• QG001 Missing check: the tool is reachable and nothing in the rules relates it to its check, so a conversation that reaches it reaches it whatever the check would say. Add the check as a condition on every route to the tool.
• QG002 Escapable guard: the check exists, and something else satisfies it: a second flow to the same tool, a branch that falls through. Apply the patch the finding names inside every flow it names.
• QG003 Multi-turn escape: a check read once and relied on later. Read the check in the same turn as the call, or carry it with the call it approved, and check again at the same depth.
• QG011 Confused deputy: a value from outside reached the model, and a route lets that model call a privileged tool with no person confirming it. Put the tool on the guarded tool step, so a call carrying the mark is confirmed or refused; mark the tool that reads the outside reads_outside where it is not.
• QG012 Cascading injection: the same, where the mark entered one agent and the call is made by another after a hand-off. Put the tool on the acting agent's guarded tool step, or confirm the hand-off.
• QG013 Irreversible before undoable: if a later step fails, the run is undone, and the irreversible step stays done; the person who approved it was not told. Move the irreversible step last, or let its confirmation say that the steps after it cannot undo it.
• QG014 Order rule escapable: a tool declared after, not_after or at_most runs where its rule is not checked. Put the tool on the guarded tool step, where order rules are checked over the session's history.
• QG015 Undeclared route: see an undeclared route.
• QG016 Unmodelled path: an attacker campaign made a privileged call on agents the hunt found clean, so what runs and what was checked disagree. Nothing this product reports is more urgent: check again from what runs, and compare the attacker's trace with the checked graph (chapter 33).
• QG105 Unrecoverable write: a write with no undo, not marked irreversible, on a route with another write; a failed run cannot undo it, and nothing says so. Declare its undo, or mark it irreversible so a person confirms it knowing that (chapter 34).

A graph agent.yaml describes puts every tool on the guarded tool step, where each call is checked and each write is confirmed, so most of these rules find nothing on it; they find what a team's own steps leave open (chapter 27). The rules on what your declarations say are below.

Reading a finding and its witness¶

A finding's witness is the run that reaches the tool: the steps, in order, that the hunt found. The findings document --keep writes holds it under detail. Here is the QG011 that E16's own graph raises on refund_card, with the fields this section reads:

{
  "rule": "QG011",
  "title": "Confused deputy",
  "severity": "high",
  "action": "refund_card",
  "control": "a confirmation",
  "obligation": "taint:refund_card:untrusted:read_inbox",
  "reproduction": "not_yet_replayed",
  "detail": {
    "witness_path": [
      "1.0 __start__",
      "1.1 model",
      "1.2 tools",
      "1.3 model",
      "1.4 tools"
    ],
    "code_flow": [
      {
        "message": "step 1.0: the graph visits __start__"
      },
      {
        "message": "step 1.1: the graph visits model"
      },
      {
        "message": "step 1.2: the graph visits tools"
      },
      {
        "message": "step 1.3: the graph visits model"
      },
      {
        "message": "step 1.4: the graph visits tools"
      }
    ],
    "fix": "Bind 'refund_card' to the guarded tool node so a call carrying the label (untrusted:read_inbox) is confirmed or refused, or add a confirmation before it."
  }
}

Read it step by step. A step's number is turn.step: 1.2 is the second step of the first turn.

1. 1.0 __start__: the person's turn arrives.
2. 1.1 model: the model reads it, and asks for a tool.
3. 1.2 tools: the tool step runs read_inbox. What it returns comes from outside, and is marked untrusted:read_inbox.
4. 1.3 model: the model reads the e-mail and asks for refund_card. Its request carries the mark.
5. 1.4 tools: refund_card runs, carrying the mark, and nothing confirms it: this team's own tool step is not the guarded one.

detail.code_flow gives one message per step, at the line that declared the step, so a code-scanning view shows the same path. The other fields:

• control: what was missing on the way -- here, a confirmation;
• obligation: what the hunt decided, named: taint:refund_card:untrusted:read_inbox is "refund_card never runs carrying untrusted:read_inbox unconfirmed";
• detail.fix: the change that closes the path;
• reproduction: not_yet_replayed until the witness is replayed (below).

The fix closes the path at its last step: a person sees the refund, with each argument and where its value came from, and the sentence that it was proposed after reading an e-mail from outside (chapter 33).

The depth of the check¶

A clean result holds to a stated depth, and the findings document states it, under scan.obligations.graph. For the shop agent:

verify("agent.yaml", keep="kept")          # as `stalwart verify agent.yaml --keep kept`

kept = json.loads(Path("kept/agent.findings.json").read_text(encoding="utf-8"))
print(json.dumps(kept["scan"]["obligations"]["graph"], indent=2))

{
  "total": 12,
  "decided": 12,
  "steps": 60,
  "turns": 3,
  "replay": "not replayed: the hunter reads the export, not the code that built it; each finding's witness_path is the run to reproduce"
}

• total and decided: how many questions the hunt asked of this graph, and how many it decided. Where they differ, the rest are listed under scan.undecided, and nothing is claimed for them.
• steps: the most steps one run may take, 60 here, the agent's step limit. Every route is searched up to it.
• turns: how many of the person's turns a route is followed over, 3 here. An approval given at one turn and spent at a later one is found within them (QG003).

So "no way through" means no way within 60 steps and three turns. Agent Graph Verification, which checks a graph you send it yourself, states the depth it searched on its [UNROLL] line (chapter 47).

Replaying a witness¶

stalwart verify --replay replays each graph finding's witness on your agents themselves, with every tool a stub that records its calls and returns fixture data, and a scripted model that makes exactly the witness's choices. The checks run as they run in production. A finding is reproduced when the run takes the witness's path and its tool is called. One that does not reproduce means the hunt and the agents that run disagree: it is listed first, and fails the gate. Each replay is recorded as digests and its outcome, never its trace.

What you can declare about your tools¶

Each declaration is optional, and each is read by one or more rules. A rule whose declaration is missing raises nothing -- see A missing declaration hides its rule.

In agent.yaml the words go under a tool's name; in Python they are arguments of @tool(...). An MCP server's registration is a tools entry that names the server and the tools it serves, with the server's version or digest where you pin one.

At registration these declarations get four lints, none of which fails anything. Registration needs your key in QLS_API_KEY; without one the catalogue is not registered, the report's first line says so, and the lints come from the runtime as registration would give them -- the E16 run below was made that way.

Other lints read what every catalogue states rather than these declarations: a write with no result or argument schema, an irreversible tool that asks for no confirmation, two tools the selector cannot tell apart, a name the reader takes for grammar, a TypeScript function tagged neither @read nor @write, and tool code that failed the code checks. The last four are errors: the version is registered, and refused for routing until a fixed one is.

The rules that read them¶

Each rule decides one question over every path the graph or the policy allows, and states its limit: what it decides, and what it does not.

QG022, the rail with a published advisory, is in chapter 28.

Every OWASP entry a finding carries is a statement at the policy level: the part of the risk that your rules and your catalogue control. A clean scan under an entry means no finding at the policy level, not that the risk is absent. The one entry no rule covers is LLM05:2026 Data and Model Poisoning: a property of training data and model weights, which no guardrail policy decides (chapter 28).

Worked example E16: the shop agent grows¶

The shop agent takes on new tools, each with its declaration:

• run_report, whose sql argument is declared sql and is written by the model;
• get_customer, confidential: subject with subject_argument: customer_id, whose record send_receipt can carry;
• search_help, over a multi_tenant store, scoped by a tenant_id the model writes, served by an MCP server registered with no version or digest;
• remember_note, persists: notes, reachable after read_inbox, and recall_notes, which reads the notes back before refund_card;
• the billing desk, moved to a remote agent over a2a with no authentication;
• and no budget on the desk's loop.

agents:
  desk:
    model: "anthropic:claude-sonnet-5-5"
    tools: [get_order, run_report, get_customer, send_receipt, search_help, read_inbox, remember_note, recall_notes, refund_card]
    hands_off_to: [billing]
  billing:
    privilege: high
    tools: [issue_refund]
    remote: {transport: a2a, authentication: none}
tools:
  - name: get_order
    schema: {type: object, properties: {order_id: {type: string}}}
  - name: run_report
    argument_kinds: {sql: sql}
    schema: {type: object, properties: {sql: {type: string}}}
  - name: get_customer
    confidential: subject
    subject_argument: customer_id
    schema: {type: object, properties: {customer_id: {type: string}}}
  - name: send_receipt
    writes: true
    sends_out: true
    schema: {type: object, properties: {order_id: {type: string}, body: {type: string}}}
  - mcp: https://help.internal/mcp
    tools: [search_help]
  - name: search_help
    retrieves: {store: help, scope_argument: tenant_id}
    schema: {type: object, properties: {tenant_id: {type: string}, query: {type: string}}}
  - name: read_inbox
    reads_outside: true
    schema: {type: object, properties: {folder: {type: string}}}
  - name: remember_note
    writes: true
    persists: notes
    schema: {type: object, properties: {note: {type: string}}}
  - name: recall_notes
    retrieves: {store: notes}
    schema: {type: object, properties: {topic: {type: string}}}
  - name: refund_card
    writes: true
    requires_privilege: high
    schema: {type: object, properties: {order_id: {type: string}}}
  - name: issue_refund
    writes: true
    schema: {type: object, properties: {order_id: {type: string}}}
stores:
  help: {multi_tenant: true}
  notes: {persistent: true}

stalwart verify reads the file, builds the agents it describes -- the desk's model, the guarded tool step its tools are on, and a hand-off to billing -- and prints:

stalwart verify agent.yaml

The catalogue was not registered: set QLS_API_KEY to register it with the service.
Lints at registration (nothing fails on a lint):
  argument_kind_suggested (info) on search_help: query: the name suggests an argument the tool interprets; declare its kind in argument_kinds if it is one
  write_without_compensation (warning) on issue_refund: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on issue_refund: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on issue_refund: a write with no result schema, so its outcome could not be reported
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_without_compensation (warning) on remember_note: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on remember_note: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on remember_note: a write with no result schema, so its outcome could not be reported
  write_without_compensation (warning) on send_receipt: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported

Code checks: no finding in 1 file(s) (rules 4b48b0081360).

QG018 High on run_report (agent.yaml:4)
  'run_report' can run with its sql argument 'sql' written by the model or carrying a trust label, and nothing confirms the call
  OWASP LLM10:2026, LLM01:2026
  Fix: Bound 'sql' in the tool's schema with an enum, a const or a pattern; or take it from the user's typed text, the host's context or an earlier tool's result, and bind it so in the export's `arguments`; or put the call behind a confirmation -- the guarded tool node or an interrupt().

QG019 High on reply
  the reply can carry a confidential value to someone its audience does not include: a record for nobody outside the company, or one whose subject the host's context did not choose -- through the tool calls and replies the policy routes
  OWASP LLM02:2026, LLM08:2026
  Fix: Bind each confidential read's subject argument to the authenticated principal in the host's context, or remove the confidential fields from the context before the reply.

QG020 High on search_help (agent.yaml:4)
  'search_help' retrieves from the multi-tenant store 'help' with its scope taken from something other than the authenticated principal
  OWASP LLM09:2026, LLM02:2026
  Fix: Take the scope argument of 'search_help' from the host's context -- the authenticated tenant or user -- and bind it so in the export's `arguments`.

QG023 Medium on cycle:desk_model (agent.yaml:2)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on issue_refund (agent.yaml:8)
  the write 'issue_refund' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'issue_refund' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (agent.yaml:4)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on remember_note (agent.yaml:4)
  the write 'remember_note' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'remember_note' in the catalogue, or mark it irreversible.

QG105 Review on send_receipt (agent.yaml:4)
  the write 'send_receipt' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'send_receipt' in the catalogue, or mark it irreversible.

QG106 Review on search_help
  'search_help' runs code from the MCP server https://help.internal/mcp with no version and no digest: it can change without the catalogue changing
  OWASP LLM04:2026, ASI04:2026
  Fix: Pin the MCP server https://help.internal/mcp to an exact version or record its digest in the catalogue, or accept the hotspot with the reason.

QG107 Review on desk -> billing (agent.yaml:2)
  the hand-off from 'desk' to 'billing' reaches 'billing' over a2a with authentication: none: nothing shows the far side is the agent it claims to be
  OWASP ASI07:2026
  Fix: Declare and use mtls, oauth2 or signed_messages on the channel to 'billing', or accept the hotspot with the layer that authenticates it.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG022: no Colang policy and no rails config was scanned

Gate: failed

The agents agent.yaml describes put every tool on the guarded tool step, where a write runs only on its confirmation. That is why they raise no QG021 and no QG019 through send_receipt: the confirmation is the fix those rules ask for, and it is already there. A team that writes its own steps, with the same tools on a tool step that checks nothing, sees them. Here is the same catalogue with the desk's tools on such a step:

stalwart verify desk graph

QG001 High on remember_note
  'remember_note' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'remember_note' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 High on refund_card
  'refund_card' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'refund_card' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 High on send_receipt
  'send_receipt' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'send_receipt' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG011 High on refund_card
  the privileged 'refund_card' can run carrying the label untrusted:read_inbox with no confirmation or refusal
  OWASP LLM01:2026, LLM03:2026, ASI01:2026, ASI02:2026
  Fix: Bind 'refund_card' to the guarded tool node so a call carrying the label (untrusted:read_inbox) is confirmed or refused, or add a confirmation before it.

QG011 High on remember_note
  the privileged 'remember_note' can run carrying the label untrusted:read_inbox with no confirmation or refusal
  OWASP LLM01:2026, LLM03:2026, ASI01:2026, ASI02:2026
  Fix: Bind 'remember_note' to the guarded tool node so a call carrying the label (untrusted:read_inbox) is confirmed or refused, or add a confirmation before it.

QG011 High on send_receipt
  the privileged 'send_receipt' can run carrying the label untrusted:recall_notes with no confirmation or refusal
  OWASP LLM01:2026, LLM03:2026, ASI01:2026, ASI02:2026
  Fix: Bind 'send_receipt' to the guarded tool node so a call carrying the label (untrusted:recall_notes) is confirmed or refused, or add a confirmation before it.

QG011 High on refund_card
  the privileged 'refund_card' can run carrying the label untrusted:recall_notes with no confirmation or refusal
  OWASP LLM01:2026, LLM03:2026, ASI01:2026, ASI02:2026
  Fix: Bind 'refund_card' to the guarded tool node so a call carrying the label (untrusted:recall_notes) is confirmed or refused, or add a confirmation before it.

QG011 High on remember_note
  the privileged 'remember_note' can run carrying the label untrusted:recall_notes with no confirmation or refusal
  OWASP LLM01:2026, LLM03:2026, ASI01:2026, ASI02:2026
  Fix: Bind 'remember_note' to the guarded tool node so a call carrying the label (untrusted:recall_notes) is confirmed or refused, or add a confirmation before it.

QG011 High on send_receipt
  the privileged 'send_receipt' can run carrying the label untrusted:read_inbox with no confirmation or refusal
  OWASP LLM01:2026, LLM03:2026, ASI01:2026, ASI02:2026
  Fix: Bind 'send_receipt' to the guarded tool node so a call carrying the label (untrusted:read_inbox) is confirmed or refused, or add a confirmation before it.

QG018 High on run_report
  'run_report' can run with its sql argument 'sql' written by the model or carrying a trust label, and nothing confirms the call
  OWASP LLM10:2026, LLM01:2026
  Fix: Bound 'sql' in the tool's schema with an enum, a const or a pattern; or take it from the user's typed text, the host's context or an earlier tool's result, and bind it so in the export's `arguments`; or put the call behind a confirmation -- the guarded tool node or an interrupt().

QG019 High on send_receipt
  'send_receipt' sends data out of the host while the session holds a confidential value, and nothing confirms the call -- through the tool calls and replies the policy routes
  OWASP LLM02:2026, LLM08:2026
  Fix: Confirm the call to 'send_receipt' -- on the guarded tool node or with an interrupt() -- or remove the confidential fields from the context before it.

QG019 High on reply
  the reply can carry a confidential value to someone its audience does not include: a record for nobody outside the company, or one whose subject the host's context did not choose -- through the tool calls and replies the policy routes
  OWASP LLM02:2026, LLM08:2026
  Fix: Bind each confidential read's subject argument to the authenticated principal in the host's context, or remove the confidential fields from the context before the reply.

QG020 High on search_help
  'search_help' retrieves from the multi-tenant store 'help' with its scope taken from something other than the authenticated principal
  OWASP LLM09:2026, LLM02:2026
  Fix: Take the scope argument of 'search_help' from the host's context -- the authenticated tenant or user -- and bind it so in the export's `arguments`.

QG021 High on remember_note
  'remember_note' can write to the persistent store 'notes' while the session carries content from outside, and nothing confirms the write
  OWASP ASI06:2026, LLM01:2026
  Fix: Confirm the write to 'notes', strip or refuse labelled values before it, or declare the store not persistent if it is not.

QG001 Medium on get_customer
  'get_customer' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'get_customer' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 Medium on get_order
  'get_order' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'get_order' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 Medium on read_inbox
  'read_inbox' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'read_inbox' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 Medium on run_report
  'run_report' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'run_report' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 Medium on recall_notes
  'recall_notes' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'recall_notes' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG001 Medium on search_help
  'search_help' can run on a plain tool node, without the tool check
  OWASP LLM03:2026, ASI02:2026
  Fix: Bind 'search_help' to the guarded tool node (StalwartToolNode) in place of the plain tool node, so every call to it is checked.

QG023 Medium on cycle:model (run_e16.py:48)
  the cycle through 'model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 25 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on remember_note
  the write 'remember_note' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'remember_note' in the catalogue, or mark it irreversible.

QG105 Review on send_receipt
  the write 'send_receipt' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'send_receipt' in the catalogue, or mark it irreversible.

QG106 Review on search_help
  'search_help' runs code from the MCP server https://help.internal/mcp with no version and no digest: it can change without the catalogue changing
  OWASP LLM04:2026, ASI04:2026
  Fix: Pin the MCP server https://help.internal/mcp to an exact version or record its digest in the catalogue, or accept the hotspot with the reason.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG022: no Colang policy and no rails config was scanned
  QG107: no agent in the catalogue declares a remote channel

Gate: failed

The QG011 on refund_card through recall_notes comes from the store: until every write into notes is proved clean, what recall_notes reads back is untrusted, so the privileged call it feeds is a confused deputy (chapter 33).

A missing declaration hides its rule¶

The same agent with the sql kind removed from run_report:

stalwart verify agent.yaml

The catalogue was not registered: set QLS_API_KEY to register it with the service.
Lints at registration (nothing fails on a lint):
  argument_kind_suggested (info) on run_report: sql: the name suggests an argument the tool interprets; declare its kind in argument_kinds if it is one
  argument_kind_suggested (info) on search_help: query: the name suggests an argument the tool interprets; declare its kind in argument_kinds if it is one
  write_without_compensation (warning) on issue_refund: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on issue_refund: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on issue_refund: a write with no result schema, so its outcome could not be reported
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_without_compensation (warning) on remember_note: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on remember_note: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on remember_note: a write with no result schema, so its outcome could not be reported
  write_without_compensation (warning) on send_receipt: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported

Code checks: no finding in 0 file(s) (rules 4b48b0081360).

QG019 High on reply
  the reply can carry a confidential value to someone its audience does not include: a record for nobody outside the company, or one whose subject the host's context did not choose -- through the tool calls and replies the policy routes
  OWASP LLM02:2026, LLM08:2026
  Fix: Bind each confidential read's subject argument to the authenticated principal in the host's context, or remove the confidential fields from the context before the reply.

QG020 High on search_help (agent.yaml:4)
  'search_help' retrieves from the multi-tenant store 'help' with its scope taken from something other than the authenticated principal
  OWASP LLM09:2026, LLM02:2026
  Fix: Take the scope argument of 'search_help' from the host's context -- the authenticated tenant or user -- and bind it so in the export's `arguments`.

QG023 Medium on cycle:desk_model (agent.yaml:2)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on issue_refund (agent.yaml:8)
  the write 'issue_refund' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'issue_refund' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (agent.yaml:4)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on remember_note (agent.yaml:4)
  the write 'remember_note' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'remember_note' in the catalogue, or mark it irreversible.

QG105 Review on send_receipt (agent.yaml:4)
  the write 'send_receipt' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'send_receipt' in the catalogue, or mark it irreversible.

QG106 Review on search_help
  'search_help' runs code from the MCP server https://help.internal/mcp with no version and no digest: it can change without the catalogue changing
  OWASP LLM04:2026, ASI04:2026
  Fix: Pin the MCP server https://help.internal/mcp to an exact version or record its digest in the catalogue, or accept the hotspot with the reason.

QG107 Review on desk -> billing (agent.yaml:2)
  the hand-off from 'desk' to 'billing' reaches 'billing' over a2a with authentication: none: nothing shows the far side is the agent it claims to be
  OWASP ASI07:2026
  Fix: Declare and use mtls, oauth2 or signed_messages on the channel to 'billing', or accept the hotspot with the layer that authenticates it.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG022: no Colang policy and no rails config was scanned

Gate: failed

No QG018: the rule has nothing to read. Registration says a declaration may be missing (argument_kind_suggested), and the summary lists the rule as having had nothing to read. A missing declaration hides its rule -- the scan is clean of what it was not told about, and says so.

An undeclared route¶

A route is where a run may go after a step. In agent.yaml every route is declared: an agent goes to its tools, back to its model, or to an agent it names in hands_off_to, and nowhere else. A step a team writes in code can choose where to go next with a function that reads what the model said. If nothing declares the places it may choose, the hunt has to assume it can choose any step -- past every check the design puts in the way. That is QG015, and every way through it opens names it, so one fix closes them all: declare where the route may go.

The gate in CI¶

stalwart verify exits 1 when the gate fails and 3 when nothing could be checked, so a CI step that runs it stops on either. Keep the findings with --keep and upload them as the run's record; chapter 46 has the workflow.

The limits¶

• A clean scan proves that no path reaches a tool without its check, its confirmation or the step it must follow, to the depth stated. It does not prove the model chooses well, and your own functions are read as able to do anything.
• The depth is the one the findings document states: the run's step limit and the number of turns. A path longer than either is not searched.
• The hunter decides where a model-written value goes, not what your tool does with it.
• A rule whose declaration is missing raises nothing; the summary lists it as having had nothing to read.
• A declaration you make -- sandboxed, sanitised_by, an authentication on a channel -- is taken as stated and listed under the scan's assumptions; the hunter does not verify it.

45 · Guarded and proved¶

A stalwart-agents agent is guarded while it runs and proved before it runs. Two checks stand around its model, both decided by native code on your machine -- the licensed runtime the package carries, or the qls-router you run yourself -- and never by a model: the router decides whether the model is called at all, and the tool check decides every call. Before the agent's first turn, the bug hunter searches its graph for a way past either. This chapter is how the two checks decide, what that guarantees, and how the proof runs.

Two checks and a proof¶

1. The router reads every turn before any model sees it (below).
2. The tool check decides every tool call, whether the router or the model chose it (below).
3. The proof searches every route through your agents for a way to a tool without its check, its confirmation or the step it must follow (below).

The hosted API decides nothing while your agent runs. Where neither the runtime nor your router can answer, a plan or a check raises SelectorUnavailableError and the call is not made: the agent never falls back to an unchecked call.

The router: whether a model is called¶

Every turn is read into a plan against what your tools declare. Where the turn goes, generated from the router itself:

A turn nothing reads is asked to be said again once. After that, the agent's on_unread decides:

Where an agent says nothing, each conversation takes the setting of your console's Settings page at its first turn, or refuse.

When no model is ever called¶

The router answers the turns it reads without a model. A model is called only where your settings send a turn to one, so no model is ever called when all of these hold:

• the agents name no model;
• on_unread is refuse, or answer with the extract composer (chapter 44);
• no agent hands_off_to another;
• replies are your phrasings: replies: {composer: template} (chapter 42).

The tool check: every call, whoever chose it¶

The check runs in the runtime, once per call, with the catalogue, the rulebook and the session's history, and the first rule that applies decides. The rules, in the order the runtime takes them (chapter 32 gives each its example):

It gives one of three answers:

• allow: the tool runs once, with an idempotency key.
• confirm: a person is asked, shown each argument and where its value came from; the call is checked again with the signed token their approval carries (chapter 35).
• refuse: nothing runs. Where earlier steps of the run had written something, those writes are undone, last first (chapter 34).

What is guaranteed, and what is not¶

Guaranteed. A refused call runs nothing. A confirmed call runs only with the exact arguments the person saw. The history cannot be dropped, edited or reordered without the check saying history_invalid. No model is called unless your settings route a turn to one.

Not guaranteed. The check judges names, schemas, marks and the history, never what a reply means (chapter 42 holds replies to what was done). A tool reachable outside the agent can be called around the check, unless the gateway stands in front of it. Agent(..., verify=False) skips the proof.

The rulebook it runs under¶

An agent's rulebook: (Colang 1.0 or 2.x) is what its graph is proved against, and it is registered with the catalogue for the tool check to enforce, so the rules proved are the rules the agent runs under:

• stalwart verify registers the catalogue with the rulebook, compiled into the record the tool check evaluates (chapter 46).
• At start-up, before its first turn, an agent compares the newest registered version's digest of its rulebook with the SHA-256 of the file's bytes. Equal, nothing is registered; different or absent, the catalogue is registered again with the rulebook before the agent runs.
• With no rulebook:, the agent still registers its catalogue at start-up where the newest version does not hold its tools, each a read or a write as declared, so the tool check knows them.
• Where it cannot be registered -- no service, a rulebook the compiler refuses (rulebook_unsupported, naming the line), or an organisation that registers rulebooks as compiled records only (policy_input_refused) -- the agent does not run. It raises RulebookNotHeld, saying why.

The shop's two desks, with a rulebook that lets a refund run only for an order that is paid:

define user ask refund
  "refund order 4411"
  "give the customer their money back"

define flow refunds
  user ask refund
  if $order_paid
    execute refund_card
  else
    bot refuse

# The shop's two desks, with a rulebook that decides when a refund may run.
agents:
  desk:
    model: "anthropic:claude-sonnet-5-5"
    tools: [get_order, charge_card, send_receipt, read_inbox]
    hands_off_to: [billing]
    on_unread: refuse
  billing:
    privilege: high
    tools: [refund_card]
tools:
  - python: shop.tools
rulebook: ./policy.co                      # proved against, and registered for the tool check

The first start registers it; the second finds it held:

desk = Agent.from_file("agent.yaml")
desk.chat("show me order 4411", user="u-17")   # before this first turn, the rulebook is registered

with open("policy.co", "rb") as policy:
    digest = hashlib.sha256(policy.read()).hexdigest()
with Client() as client:
    held = client.catalogue()                   # the newest catalogue version
    print("version", held.version, "holds policy.co:", held.rulebook_ir["policy_sha256"] == digest)

again = Agent.from_file("agent.yaml")           # a second start of the same agent
again.chat("show me order 4412", user="u-18")
with Client() as client:
    print("version", client.catalogue().version, "after a second start")

And the tool check decides under it, with the context your application supplies:

with Client() as client:
    billing = client.session("billing")
    billing.read("refund order 4411 40.00")
    for paid in (False, True):
        check = billing.check_tool_call("refund_card", {"order_id": "4411", "amount": 40.00},
                                        origins={"order_id": "typed", "amount": "typed"},
                                        context={"order_paid": paid})
        print(f"order_paid {paid}:", check.decision, ", ".join(check.reasons), check.detail.get("flow", ""))

version 1 holds policy.co: True
version 1 after a second start
order_paid False: refuse refuse refunds
order_paid True: confirm write 

The unpaid order's refund is refused by the rulebook's refunds flow; the paid one passes the rulebook and is confirmed, as every write is.

A rulebook's actions name tools by execute <tool>, as the refunds flow does. A bot line is something the bot says, not a tool call: a flow that only says bot execute_refund_card governs no tool.

The proof¶

The agents compile to a graph (qls-graph/1) and its catalogue. Each obligation -- this tool never runs without its check, this write never runs unconfirmed after reading from outside -- is a question the hunter answers over every route:

• reachable: a finding, with a witness path mapped back to the line of agent.yaml (chapter 31);
• unreachable: the obligation holds, to the depth stated (chapter 31);
• undecided: reported as undecided, never as clean.

It runs three ways:

• Automatically. Agent(..., verify=True), the default, proves the graph on our service before the first turn, once per graph: a rebuild of the same graph is answered from the cache. A finding of type Vulnerability raises NotVerified, with the findings, and nothing runs. A proof that could not run raises too, never a clean result.
• As a CI gate. stalwart verify agent.yaml registers, hunts and prints the report, and exits 1 when the gate fails (chapter 46). The shop agent of chapter 30:

$ stalwart verify agent.yaml
stalwart verify agent.yaml

Registered as catalogue version 1, with the rulebook policy.co: the tool check enforces the rules verified here.
Lints at registration (nothing fails on a lint):
  write_not_retry_safe (warning) on charge_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on charge_card: a write with no result schema, so its outcome could not be reported
  compensation_not_retry_safe (warning) on charge_card: 'refund_card' undoes it and is not retry-safe
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported
  compensation_out_of_reach (warning) on charge_card: agent 'desk' can call it but not 'refund_card', which undoes it, so a run of that agent cannot undo it

Code checks: no finding in 4 file(s) (rules 4b48b0081360).

QG023 Medium on cycle:desk_model (agent.yaml:3)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card (agent.yaml:10)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (policy.co:19)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG019: no tool is declared confidential
  QG020: no retrieval from a store declared multi-tenant
  QG021: no tool is declared to write a store declared persistent
  QG106: no tool declares a source or a server, and the catalogue declares no modules
  QG107: no agent in the catalogue declares a remote channel

Gate: passed
$ echo $?
0

• Over the API. Client().verify_graph(graph, catalogue=..., rulebook=..., session_steps=...) sends the graph desk.export() returns and the catalogue desk.catalogue() returns, as a graph job (chapter 47).

The rules it proves¶

Every rule the hunt of an agent's graph and rulebook raises, generated from the hunter's catalogue. Fails the gate is what stalwart verify holds a build to; stops Agent(...) is what the automatic proof refuses to run with:

Each rule's page says what it proves and what it does not.

The limits¶

• A model chooses freely. The proof reads a model step as able to choose any tool and any argument it is given; the checks hold what it chooses, and nothing proves its choices good.
• Steps that are not ours are opaque. A step your team writes is read as able to do anything its declared routes allow (chapter 31).
• A clean proof holds to the depth it states: the run's step limit and the number of turns the findings document names.
• The proof covers the graph, the catalogue and the rulebook, not your tools' own code; the code checks of chapter 46 read that code for network and credential problems only.
• The rulebook decides only the tools its flows call by execute. A tool no flow governs is decided by the rest of the check.

46 · Checking an agent in CI: stalwart verify¶

stalwart verify agent.yaml is the check a build fails on. It is the proof an agent runs before its first turn (chapter 45), made into a command with an exit status, so a pull request that opens a way past a check, or adds code that turns a certificate check off, stops before it is merged. This chapter is the command in full: what it runs and where, a workflow, a passing and a failing report, and every option and exit status.

Four steps, one on your machine¶

1. The code checks, on your machine. Every Python, JavaScript, TypeScript and Go file of the folder agent.yaml is in is read for network-security and credential problems, by the runtime the package carries. Any finding fails the gate. Your code never leaves the machine: only the count of findings by rule goes with the registration.
2. The registration. The catalogue -- every tool with its words -- is registered as a new version, with the file's rulebook: compiled into the record the tool check enforces. The rules checked here are the rules your agent runs under (chapter 45).
3. The hunt, on our servers. The agents' graph, the catalogue and the rulebook are sent as one hunt job (POST /v1/solves, with its quality gate on), and the bug hunter searches every route through your agents. The solver runs on Stalwart's servers and never on your machine, and nothing besides the package is installed.
4. The report. Each finding with its rule, its severity, the line of agent.yaml it comes from, its OWASP entries and its fix, then Gate: passed or Gate: failed. Chapter 31 reads a report line by line.

A hunt that could not run -- no route to the service, a job that ended without a findings document -- is an error with exit status 3. It is never a clean result.

What it needs¶

Your key in QLS_API_KEY, and a licence activated on the machine, as every command but stalwart licence, stalwart doctor and stalwart check needs. A CI runner is a new machine every time, so the job activates the licence first and releases its place last, whatever happened in between: a released place is free for the next run at once (chapter 38).

A GitHub Actions workflow¶

# .github/workflows/agent.yml: the shop agent checked on every pull request.
name: agent
on: [pull_request]
jobs:
  verify:
    runs-on: ubuntu-latest
    env:
      QLS_API_KEY: ${{ secrets.QLS_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
      - run: pip install stalwart-agents
      - run: stalwart licence activate          # a runner is a new machine every time
      - run: stalwart verify agent.yaml --replay --keep findings/
      - if: always()                            # the place is released whatever happened above
        run: stalwart licence release
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: findings
          path: findings/

QLS_API_KEY is a repository secret holding a key of your account (chapter 5). The release step and the upload carry if: always(), so they run after a failed check too: the place is freed, and the findings --keep wrote are kept with the run.

A passing report¶

The shop agent of chapter 30, checked as the workflow checks it:

$ stalwart verify agent.yaml
stalwart verify agent.yaml

Registered as catalogue version 1, with the rulebook policy.co: the tool check enforces the rules verified here.
Lints at registration (nothing fails on a lint):
  write_not_retry_safe (warning) on charge_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on charge_card: a write with no result schema, so its outcome could not be reported
  compensation_not_retry_safe (warning) on charge_card: 'refund_card' undoes it and is not retry-safe
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported
  compensation_out_of_reach (warning) on charge_card: agent 'desk' can call it but not 'refund_card', which undoes it, so a run of that agent cannot undo it

Code checks: no finding in 4 file(s) (rules 4b48b0081360).

QG023 Medium on cycle:desk_model (agent.yaml:3)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card (agent.yaml:10)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (policy.co:19)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG019: no tool is declared confidential
  QG020: no retrieval from a store declared multi-tenant
  QG021: no tool is declared to write a store declared persistent
  QG106: no tool declares a source or a server, and the catalogue declares no modules
  QG107: no agent in the catalogue declares a remote channel

Gate: passed
$ echo $?
0

The findings here are a review hotspot and a medium finding under rules that do not gate (chapter 31), so the gate passes and the command exits 0.

A failing report¶

A teammate adds a helper to the same project that fetches a carrier's tracking page, with the certificate check turned off:

"""A helper a teammate added to the shop's project: the carrier's tracking page for an order."""

import httpx


def tracking_page(order_id: str) -> str:
    """Fetch the carrier's tracking page for an order."""
    page = httpx.get(f"https://tracking.carrier.example/{order_id}", verify=False)
    return page.text

It is not a tool, and no agent calls it. The code checks read every file of the project all the same, since a tool can import any of them:

$ stalwart verify agent.yaml
stalwart verify agent.yaml

Registered as catalogue version 2, with the rulebook policy.co: the tool check enforces the rules verified here.
Lints at registration (nothing fails on a lint):
  write_not_retry_safe (warning) on charge_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on charge_card: a write with no result schema, so its outcome could not be reported
  compensation_not_retry_safe (warning) on charge_card: 'refund_card' undoes it and is not retry-safe
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported
  compensation_out_of_reach (warning) on charge_card: agent 'desk' can call it but not 'refund_card', which undoes it, so a run of that agent cannot undo it

Code checks: 1 finding(s); each one fails, and only changing the code clears it.
  QC002 CWE-295 shop/tracking.py:8  This code turns TLS verification off (`verify=False`), at shop/tracking.py:8.
      shop/tracking.py:8  turns TLS verification off (`verify=False`)

QG023 Medium on cycle:desk_model (agent.yaml:3)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card (agent.yaml:10)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (policy.co:19)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG019: no tool is declared confidential
  QG020: no retrieval from a store declared multi-tenant
  QG021: no tool is declared to write a store declared persistent
  QG106: no tool declares a source or a server, and the catalogue declares no modules
  QG107: no agent in the catalogue declares a remote channel

Gate: failed
$ echo $?
1

QC002 names the file and the line, and the gate fails with exit status 1. Removing verify=False clears it; nothing else does. A code-check finding has no warning level, no comment that waives it and no option that skips it.

The same check from Python¶

For a pipeline that is not a shell, verify is the same check, and render the same report:

from stalwart_agents import render, verify

result = verify("agent.yaml")
print(render(result), end="")
print("exit status:", result.exit_code)      # 0 the gate passed, 1 it failed

stalwart verify agent.yaml

Registered as catalogue version 3, with the rulebook policy.co: the tool check enforces the rules verified here.
Lints at registration (nothing fails on a lint):
  write_not_retry_safe (warning) on charge_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on charge_card: a write with no result schema, so its outcome could not be reported
  compensation_not_retry_safe (warning) on charge_card: 'refund_card' undoes it and is not retry-safe
  write_without_compensation (warning) on refund_card: a write neither compensated nor marked irreversible
  write_not_retry_safe (warning) on refund_card: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on refund_card: a write with no result schema, so its outcome could not be reported
  write_not_retry_safe (warning) on send_receipt: a write with no idempotency, so an ambiguous failure goes to a person
  write_without_result_schema (warning) on send_receipt: a write with no result schema, so its outcome could not be reported
  compensation_out_of_reach (warning) on charge_card: agent 'desk' can call it but not 'refund_card', which undoes it, so a run of that agent cannot undo it

Code checks: no finding in 4 file(s) (rules 4b48b0081360).

QG023 Medium on cycle:desk_model (agent.yaml:3)
  the cycle through 'desk_model' calls the model or a tool, and nothing declares a budget: the framework's recursion limit of 60 is the run's only ceiling
  OWASP LLM06:2026, ASI08:2026
  Fix: Declare a budget for the agent or the run, or add an exit the model cannot avoid -- a counter in the state, declared on the edge as exit_after.

QG105 Review on refund_card (agent.yaml:10)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

QG105 Review on refund_card (policy.co:19)
  the write 'refund_card' has no compensation, is not irreversible, and runs on a path with another write
  OWASP LLM03:2026, ASI08:2026
  Fix: Declare a compensation for 'refund_card' in the catalogue, or mark it irreversible.

Rules with nothing to read, so nothing raised:
  QG017: no argument is declared as code or a shell command
  QG018: no argument is declared as SQL, a query, a path, a URL or a template
  QG019: no tool is declared confidential
  QG020: no retrieval from a store declared multi-tenant
  QG021: no tool is declared to write a store declared persistent
  QG106: no tool declares a source or a server, and the catalogue declares no modules
  QG107: no agent in the catalogue declares a remote channel

Gate: passed
exit status: 0

result.exit_code is what the command exits with: 0 where the gate passed, 1 where it failed. A file that cannot be read, a missing key or licence, and a hunt that could not run are raised as AgentFileError, LicenceRequired and HunterUnavailable (chapter 30).

The options¶

Generated from the command's own parser:

--keep DIR writes agent.graph.json, agent.catalogue.json and agent.findings.json; stalwart redteam reads the last (chapter 33). --export-only FILE registers nothing and hunts nothing.

The exit statuses¶

Generated from the command itself:

For verify, 2 is a file that cannot be read, or no key or no licence; 3 is a hunt or a code check that could not run, so nothing was checked.

The limits¶

• Chapter 45's limits hold here (chapter 45): the model is read as choosing anything, your own functions as able to do anything, and a clean hunt holds to the depth it states (chapter 31).
• A hunt that could not run is never a clean result. The hunt is the service's, so a build with no route to api.stalwart.it cannot pass the gate: it exits 3, never 0.
• The code checks read network security and credential handling, in Python, JavaScript, TypeScript and Go, and nothing else about what your code does.

32 · What every tool call is checked against¶

Every call your agent makes to a tool is checked before it runs, whoever asked for it: the person's turn, the model, or an undo after a failure. The check answers allow, confirm (a person must approve it first) or refuse, with the reasons. This chapter is the check's order, and what each step means.

Where the check runs¶

The check runs in the licensed runtime on your machine, with the catalogue and the rulebook you registered (chapter 31). Your agent calls it at its guarded tool step, once per call, and acts on the answer: it runs an allowed call once, waits for a person on confirm, and runs nothing on refuse. Nothing about the call -- its tool, its arguments, the turn -- is sent to us to decide it.

The order¶

The steps are taken in this order, and the first that answers ends the check. Generated from the runtime's own check:

So a refusal always names the first thing wrong: a call to a tool the catalogue does not have is refused for that, whatever its arguments; a call that passes steps 1 to 8 with a valid approval is allowed at 9, without asking again; and a write that reaches step 10 is confirmed, with every reason at once.

One call, every check¶

The shop's refund, asked about in each of the ways a run can ask. The billing desk's session reads the person's turn, refund order 4411 40.00, and each call is checked with that session's history:

def show(what, check):
    detail = {k: v for k, v in check.detail.items() if k in ("pointer", "rule", "destinations", "wording")}
    print(f"{what}: {check.decision} {', '.join(check.reasons)} {detail if detail else ''}".rstrip())


with Client() as client:
    billing = client.session("billing")       # one conversation with the billing desk
    billing.read("refund order 4411 40.00")   # what the person typed

    refund = {"order_id": "4411", "amount": 40.00}
    show("the model wrote the amount",
         billing.check_tool_call("refund_card", refund, origins={"order_id": "typed", "amount": "model"}))
    asked = billing.check_tool_call("refund_card", refund, origins={"order_id": "typed", "amount": "typed"})
    show("the person typed the amount", asked)
    show("the person approved it",
         billing.check_tool_call("refund_card", refund, origins={"order_id": "typed", "amount": "typed"},
                                 confirmation=asked.confirmation))

    asked = billing.check_tool_call("refund_card", refund, origins={"order_id": "typed", "amount": "typed"})
    show("approved, then the amount changed",
         billing.check_tool_call("refund_card", {"order_id": "4411", "amount": 45.00},
                                 origins={"order_id": "typed", "amount": "model"}, confirmation=asked.confirmation))

    show("an argument the tool does not declare",
         billing.check_tool_call("refund_card", {**refund, "note": "goodwill"},
                                 origins={"order_id": "typed", "amount": "typed", "note": "model"}))
    show("a tool the catalogue does not have", billing.check_tool_call("delete_order", {"order_id": "4411"}))
    show("a destination in a tool that sends nothing out",
         billing.check_tool_call("refund_card", {"order_id": "https://refunds.invalid/4411", "amount": 40.00},
                                 origins={"order_id": "model", "amount": "typed"}))
    show("proposed after reading the inbox",
         billing.check_tool_call("refund_card", refund, origins={"order_id": "typed", "amount": "typed"},
                                 labels={"call": ["untrusted:read_inbox"]}))

    billing.history.entries[0]["tool"] = "get_order"     # the history, edited by hand
    show("the history edited by hand",
         billing.check_tool_call("refund_card", refund, origins={"order_id": "typed", "amount": "typed"}))

the model wrote the amount: refuse refuse
the person typed the amount: refuse refuse
the person approved it: refuse refuse
approved, then the amount changed: refuse refuse
an argument the tool does not declare: refuse argument_schema {'pointer': '/note', 'rule': 'additionalProperties'}
a tool the catalogue does not have: refuse unknown_tool
a destination in a tool that sends nothing out: refuse undeclared_destination {'destinations': ['https://refunds.invalid/4411']}
proposed after reading the inbox: refuse refuse
the history edited by hand: refuse history_invalid

Line by line:

• the amount the model wrote asks the person, naming the argument: argument_origin: amount;
• the amount the person typed -- it is in their turn -- still asks, because every write is confirmed: the reason is just write;
• presented with the token from that confirmation, the same call is allow, confirmed;
• an approval presented after the amount changed covers nothing: the check asks again;
• note, an argument refund_card does not declare, is refused at /note;
• delete_order is not in the catalogue;
• a web address in the order number is a destination, and refund_card does not send anything out;
• a refund proposed after reading the inbox asks, and its question says so;
• a history edited by hand is refused before anything else is read.

Where each argument came from¶

Every argument of a write carries its origin, and the check trusts some origins and not others:

• typed: the person typed it. Trusted only where the value is in their turns, as they typed it: a value claimed as typed that no turn holds is asked about;
• reply, context and computed: the person's answer to a question, a value your application put in the run's context (such as the signed-in customer), and a calculation the run made. Trusted;
• result:<step> or result:<step>.<field>: a field of an earlier call's result. Trusted where that call ran in this session and its tool's result schema declares the field;
• model, and anything else: the model wrote it. Asked about, by name.

An undeclared argument is refused. A tool's schema says what its arguments are, and an argument it does not declare is one nobody wrote a rule for: a note on a refund, a cc on a receipt. Refusing it at step 3 means the rules that follow are never asked about a value they cannot see.

The session's history¶

The check reads the session's history: every call checked so far, with its step, its tool, its agent, the digests of its arguments and their origins, the marks it carried, the check's answer, and what became of it. The caller carries the history and we sign it: each answer adds the call's entry and a token that signs it together with every entry before it, with your licence's keys. The next check verifies those signatures.

history_invalid means the signatures did not verify: an entry was edited, removed or reordered, or the history is another session's. Nothing else is read, and the answer starts a new history at step 0. What became of a call -- ran, failed, outcome_unknown or compensated -- is the caller's report, and the next entry's link carries it.

Order rules¶

A tool's order words are checked over the verified history (chapter 30):

• after: [get_order]: the tool runs only once a get_order ran in this session. A call that was allowed but failed does not count. An after can bind an argument to that earlier call's result, so a refund names a charge this session made, not one the model wrote: the argument's origin must be that call's result, or the check refuses it naming the binding;
• not_after: [read_inbox], or not_after: [untrusted]: the tool never runs once the named tool was allowed, or once anything from outside was read. A read counts whatever became of it: what it returned was read;
• at_most: 1: the tool runs at most once in the session.

Each is refused as order_rule, with the rule it broke in the detail, in the words the hunt's QG014 uses.

Undos bound to what they undo¶

An undo a failed run sends is checked like any call, and also against the step it undoes: that step ran, the undo is the one its tool declares, each argument the undo binds comes from that step's arguments or result, and the step has not been undone already. An undo that passes is allowed without a confirmation, unless your rulebook asks a question about it: the person approved the write it undoes. One sent twice is compensation_spent; one that does not match its step is compensation_mismatch (chapter 34).

The rulebook's gate¶

Your rulebook decides the call at step 7, as it was compiled at registration. A flow that refuses the call refuses it, naming its rule. A flow that asks for a question makes the call a confirmation, as gate_clarify.

Untrusted values¶

A value from a tool that reads_outside, or from an agent with less privilege than the tool requires, carries a mark (chapter 33). A write carrying a mark less trusted than the tool is held: by default it is confirmed, its question saying where the mark came from; where your organisation's taint_policy is refuse, it is refused. Reads are never held for a mark.

Confirmations and their tokens¶

A confirm comes with a token that covers exactly that call: the tool, the digest of its arguments, the session and the point of the history it was asked at, and an expiry (by default a day). The person's approval sends the call back with the token, and the check allows it at step 9. A changed argument asks again: its digest is no longer the one the token covers, so the token covers nothing, and the call is checked from the start. Once the approved call has run, the history has moved on and the token covers nothing either.

An irreversible write in a run whose earlier writes would be undone if it failed adds irreversible_pivot, and its question ends: Nothing undoes this step, and the steps after it cannot undo it.

The limits¶

• The check trusts the caller's report of whether a call ran. It decides whether a call may run; what became of it is the host's report, carried in the history. The gateway sees it, because the gateway makes the call itself.
• What stays on your machine: your agents' conversations, their tool calls and results, a calculation's data, function and result, and the documents your agents answer from and their index. We keep no session, no tool call's arguments and no tool result, and no turn's words except those a feedback record carries. (chapter 16)
• A confirmation proves a person approved the call shown, with those arguments. It does not prove they read it.
• The check governs the calls your agent makes. A tool reachable another way can be called around it (chapter 37).

33 · Attacks from documents, e-mail and other agents¶

An indirect instruction is text your agent reads -- in an e-mail, a web page, a document, another agent's hand-off -- that tells the model to do something nobody asked for. The person never typed it: it arrived inside something a tool returned. The shop agent's read_inbox returns the newest support e-mail, and the e-mail says refund order 4411 in full. A model that reads it may do what it says, and the refund is a write to someone's card. This chapter is how the product follows such text, what it does when a call carries it, and how the hunt and the attacker campaigns look for it before launch.

What reads the outside¶

Two things bring text in that nobody you trust wrote:

• A tool that reads the outside world. Mark it reads_outside=True (reads_outside: true in agent.yaml): an inbox, a web fetch, an uploaded document. What it returns is untrusted.
• An agent with less privilege. Each agent has a privilege: low, standard (the default) or high. A tool requires a privilege: the one its requires_privilege names; otherwise high for a write that only high agents reach, and standard for anything else. The shop's desk is standard and hands refunds to billing, which is high: what the desk hands over is less trusted than refund_card requires.

How a mark follows a value¶

The product puts a mark on such text and follows it, conservatively:

1. the result of a reads_outside tool is marked untrusted:<tool>, such as untrusted:read_inbox;
2. once a marked message is in what a model reads, everything that model writes carries the mark -- it has read it -- and so does every tool call it chooses, and every argument of those calls the model wrote;
3. a hand-off carries the marks of everything handed over, and the handing agent's own mark, agent:<name>;
4. a mark is removed only by removing the marked message from what the model reads -- or by a rewind, which removes it (chapter 34);
5. a calculated value carries the marks of what it was calculated from.

The person's own words carry no mark: a turn the selector reads into a tool call is the person's, held by your rules and the check. The marks are kept beside the messages, never in them, so nothing of ours is sent to a model provider.

What a marked call does¶

The check compares each mark with what the tool requires (chapter 32). An untrusted mark is below every agent; an agent: mark is that agent's privilege. A write carrying a mark below what its tool requires is held, as your organisation's taint_policy says:

• confirm, the default: a person is asked, and the question says where the mark came from. From the shop's refund proposed after reading the inbox (chapter 32): Run refund_card? This call was proposed after reading read_inbox from outside. A mark from a hand-off reads a hand-off from desk;
• refuse: the call is refused, tainted_call, and runs nothing.

A read is never held for a mark: reading is how the agent finds things out.

The confused deputy and the cascade¶

Before launch, stalwart verify asks the same question of every route: can a privileged write run carrying a mark, with nobody confirming it?

• QG011 Confused deputy: the mark entered the agent that makes the call. The privileged tool is the deputy: it acts with the agent's authority on an instruction the agent should not have taken.
• QG012 Cascading injection: the mark entered one agent, and the call is made by another after a hand-off -- the desk reads the e-mail, hands it to billing, and billing refunds.

On agents agent.yaml describes, every write is on the guarded tool step, so a marked write is always confirmed or refused and neither rule finds anything; they find what a team's own steps leave open. Chapter 31 reads a QG011 witness step by step.

Text from outside, written to memory¶

A tool that reads_outside returns untrusted text, and the hunter marks every value that carries it. When a write to a persistent store can run with such a value and no confirmation, QG021 is raised: the text is stored where a later session reads it as its own, and the mark that said it came from outside is gone.

So a store's reads are untrusted until every write into it is proved clean. The findings document stalwart verify --keep writes gives each persistent store's trust, under scan.stores:

• proved clean: every write into the store is confirmed or carries nothing marked, so what the store's reads return is trusted;
• treated as untrusted: some write is not proved clean, so every read from the store -- a recall_notes, a retrieval -- is an untrusted source, and a privileged call it feeds is a QG011.

The fix is a confirmation on the write, or stripping what is marked before it; once the write is proved clean, the store is too, and the findings its reads caused go with it. E16 shows both.

Attacker campaigns¶

The hunt finds where a way through exists. Whether a model takes it is a different question, and stalwart redteam asks it. A campaign runs your agents on your machine as they run in production -- the selector, the checks, the confirmations, each agent on its own model -- with every tool replaced by a stub that records its calls and returns fixture data. No campaign can charge a card or send a message.

$ stalwart redteam agent.yaml --findings kept/agent.findings.json --open

kept/agent.findings.json is what stalwart verify agent.yaml --keep kept wrote. Every option:

• An aimed campaign takes one finding the hunt raised, with its witness, and tries to make it happen. Each rule's goal is the call its finding is about, carrying what the rule says it must not carry:

• An open campaign has no witness: it aims at each privileged tool -- those marked requires_privilege: high, or every write where none is -- through every channel.

Three roles take turns, up to the round cap:

With --attacker provider:model, the planner and the exploiter write with your model and your key; without it they run offline, from fixed text. Each campaign is recorded as its outcome, its rounds and digests of the planted text and the trace -- never the text.

An open campaign at the shop agent, whose model does whatever the last thing it read tells it to:

status = cli.main(["redteam", "agent.yaml", "--open", "--rounds", "2"])   # stalwart redteam agent.yaml --open --rounds 2
print("exit status:", status)

open open: not realised in budget (2 round(s)): this says nothing about whether the graph is safe; last round: the model did not call 'refund_card'
exit status: 0

The planner tried the inbox, then the person's own turn. In the last round the planted text was the person's turn: the desk's selector found no tool that covers it and refused it (on_unread: refuse), so no model read it and refund_card was never called. With the confirmation the guarded tool step puts on every write, a refund the model did propose would wait for a person, and a campaign does not approve it.

Reading the result¶

• Realised: the goal call was made, with what it had to carry, and the trace shows it. stalwart redteam exits 1.
• Not realised in budget: no round realised it. This says nothing about whether the graph is safe -- a better plan, another channel or more rounds might -- and is worded so.
• QG016: an attack realised on agents the hunt called clean. Give --clean when stalwart verify found no vulnerability: a realised open campaign is then QG016, the hunt and the agents that run disagree, it is printed first, and it fails the gate. --clean is refused where the findings hold a vulnerability.

The limits¶

• Not realised in budget says nothing about safety. A campaign is a search within its rounds, by its planner; whether a way exists, to its depth, is what the hunt decides (chapter 31).
• Every tool in a red-team run is a stub. A campaign shows what your agents and your model do with a planted text, not what your tools do with the call.
• A mark is removed only by removing what carried it. The product does not judge whether marked text is harmless.
• A store's reads stay untrusted until every write into it is proved clean, which the hunter decides over the writes the graph routes; a store poisoned by a path outside the agent is not found.
• The gateway's rule is coarser, and session-wide: the gateway cannot see what the model read, so once a reads_outside tool has returned in a session, every later call in that session carries its mark (chapter 37).

34 · When a run fails: undo, retry and rewind¶

A run that writes can fail half-way: the card is charged, the CRM updated, and then the provisioning service is down. With runs: atomic, the default, the run undoes what it did, last write first, and gives one reply that says what failed, what was undone and what could not be. This chapter is how you declare an undo, what a failed run does and in which order, and what happens when nobody knows whether a write took effect, or an undo fails.

Declaring an undo¶

A write names the tool that undoes it:

@tool(writes=True, undo="refund_card")

In agent.yaml, undo: refund_card under the tool's name says the same. The undo is a tool like any other: it must be one of the same agent's tools, and it is checked before it runs.

Each argument of the undo is bound to the write it undoes: $args.<name> is an argument of the write, and $result.<field> a field of what the write returned. Before an undo runs, the check verifies that each bound argument came from that step, that the step ran, and that it was not undone already (chapter 32). So an undo refunds the charge this run made, with its amount, and never an amount the model wrote.

Irreversible steps¶

A write nothing can undo says so: @tool(writes=True, irreversible=True), such as sending a receipt. Every write is confirmed by a person before it runs (chapter 32), and an irreversible one is confirmed with a warning:

• the interface shows Nothing can undo this step once it runs. under it (chapter 35);
• in a run whose earlier writes would be undone if a later step failed, the question also ends: Nothing undoes this step, and the steps after it cannot undo it.

The person approves it knowing that if the run fails later, this step stays done. A rollback leaves it in place and lists it as not undone. Before launch, QG013 finds a route where an irreversible step can come before steps that would be undone, without its confirmation saying so.

The lints¶

Registration reads every write for what recovery needs, and says what is missing. None fails anything:

A write with neither an undo nor irreversible is also QG105 at stalwart verify, on a route with another write: a failed run cannot undo it, and nothing says so.

What a failed run does¶

A run fails when a tool raises past its retries, when the check refuses a later step of a run that has already written, when a person answers cancel all, or when a person says a write of unknown outcome did not happen. Then, in this order:

1. The model is not called again. The run goes to its rollback; nothing more is asked of any model.
2. The writes are undone, last first. Each undo is checked as the agent whose write it undoes would be checked, with the step it undoes named, and is not confirmed again: the person approved the write. The check and the sending are separate steps, so what the check allowed is recorded before anything is sent.
3. Irreversible writes are skipped, and listed.
4. The run is rewound once nothing remains to undo (below), and the person gets one reply.

The failed order¶

The shop's desk grown by a CRM update and a provisioning step, each write with its undo, and a provisioning service that is down:

# The shop's desk, grown: an order is charged, marked paid in the CRM, and its account provisioned.
agents:
  desk:
    model: "anthropic:claude-sonnet-5-5"
    tools: [get_order, charge_card, refund_card, update_crm, restore_crm, provision_account, send_receipt]
    on_unread: refuse
tools:
  - python: shop.tools
rulebook: ./policy.co
runs: atomic                               # a failed run is undone, last write first

desk = Agent.from_file("agent.yaml")


def order(turn, user):
    reply = desk.chat(turn, user=user)
    while reply.waiting and reply.waiting[0].kind == "confirm":
        reply = desk.confirm(user, "approve")       # the person approves each write as it comes
    return reply


reply = order("charge order 4411 40.00, then update the crm for order 4411, "
              "then provision the account for order 4411", "u-17")
print(reply.text)
print("The tools ran in this order:")
for number, name in enumerate(CALLS, 1):
    print(f"{number}. {name}")

Then the same order with a receipt sent before provisioning:

CALLS.clear()
reply = order("charge order 4411 40.00, then send the receipt for order 4411, "
              "then provision the account for order 4411", "u-18")
print(reply.text)

This run stopped: provision_account failed (RuntimeError). Undone, latest first: update_crm (by restore_crm), charge_card (by refund_card).
The tools ran in this order:
1. charge_card
2. update_crm
3. provision_account
4. restore_crm
5. refund_card
This run stopped: provision_account failed (RuntimeError). Undone, latest first: charge_card (by refund_card). Not undoable, and left as they are: send_receipt.

In the first run the tools ran in this order, as the example printed it and as it asserts:

1. charge_card
2. update_crm
3. provision_account
4. restore_crm
5. refund_card

The CRM is restored before the card is refunded: last write first. The run's three steps came from the person's one turn, read by the selector; the model was never called, and the person got one reply. In the second run the charge is refunded and the receipt is listed as not undone: it was confirmed as irreversible, and it stays sent.

Skip this step and cancel all¶

A confirmation takes three answers (chapter 35):

• approve: the call runs, with exactly the arguments shown;
• skip: the call does not run, its result says it was declined, and the run goes on without it;
• cancel all: the run fails, as a person cancelling it, and what it already did is undone as above.

Retries and their keys¶

Every write carries an idempotency key: SHA-256 over the session, the step, the tool and the arguments, so the same call at the same step of the same session always has the same key. A retry, a resumed run and a re-sent undo all carry the key the first try carried, so a tool that honours keys does the thing once.

A call is recorded in flight before it is sent. A read, or a write marked retry_safe=True, is tried again under the same key -- at most twice more -- when it fails in a way that leaves its effect in doubt, such as a timeout or a lost connection. Any other write is sent once.

An outcome nobody knows¶

A write that is not retry-safe can fail in a way that leaves its effect in doubt: the call was sent and no answer came back. Sending it again could charge twice, and assuming it failed could leave a charge nobody undoes, so the run does neither. It asks a person: Did charge_card take effect? The call was sent and no answer came back. The answers are happened, and the run goes on as if the call ran, or did not happen, and the run fails and is undone.

A process that stops mid-call finds the in-flight record when the run resumes, on any worker. A retry-safe call is sent again under its key; any other is the same question: Did charge_card take effect? It was being sent when the run stopped.

An undo that fails¶

An undo that raises is tried again under its own key, three more times, waiting longer between tries. Then the rollback stops: undoing the writes below it out of order could leave things worse than either state. The run is stuck, and a person is asked: Undoing <tool> failed after <tries> tries (<error>). <n> step(s) still need undoing. The answers:

• retry: the undo is sent again, under its key;
• done by hand: someone undid it outside the agent; the rollback goes on with the next write;
• abandon: the rollback stops here, and the reply lists what still needs undoing.

The rewind¶

Once nothing remains to undo, the run's own messages -- everything after the person's turn that started it -- are taken out of what the model reads next, and one summary reply is added. The next turn does not read an error trace, or a marked e-mail the failed run read.

• The rewind waits for every undo. While any undo remains, or the run is stuck, it is refused (rewind_with_pending_compensation): forgetting what still needs undoing is the one mistake a rewind must not make.
• The record is never rewound. The session's history keeps every call, every undo and every outcome, so an order rule or a cap still sees that a charge ran and was refunded. The memory is rewound; the record is not.

An operator can undo a thread's run from outside it, with the operator token (chapter 35).

The limits¶

• An irreversible step is not undone. It is confirmed with a warning, and listed as not undone.
• A run compensates; it does not lock. Between a write and its undo, the charge is real: anyone who looks sees it, and anything that acts on it acts. An undo is a second write, not a step back in time.
• The rewind waits for every undo, and the record is never rewound.
• The undo is your function, and does what it does. The product decides when it is called and with which arguments, and checks the call; whether it really refunds the card is your code's.
• A write of unknown outcome is decided by a person; the product does not guess.

35 · People in the loop¶

Some calls wait for a person: every write, before it runs; a write nobody knows the outcome of; an undo that keeps failing. This chapter is what that person sees and answers, how an answer reaches the run wherever it waits -- on another worker, from another channel -- and the interface that shows it all, @stalwart-agents/agent-ui, which stalwart serve gives you before you write any of it.

A confirmation, from the person's side¶

The shop agent's charge from chapter 30, as it waits:

{"card": "visa-0019", "customer": "c-208", "order_id": "4411", "status": "unpaid", "total": "40.00"}
charge_card | Run charge_card? | ['approve', 'skip', 'cancel_all']
   {'name': 'amount', 'value': 40, 'origin': 'typed'}
   {'name': 'order_id', 'value': '4411', 'origin': 'typed'}
{"amount": "40", "charge_id": "ch-4411", "order_id": "4411"}

A confirmation shows:

• the call: the tool and its question -- Run charge_card?, or the wording your tool gives;
• each argument, with where its value came from: typed (the person typed it), reply (they answered a question), context (your application gave it, such as the signed-in customer), computed, result:<step> (an earlier step returned it), or model (the model wrote it). The interface says these in words: you typed it, the assistant wrote it;
• the reasons it waits, as the check gave them: write, argument_origin: amount, tainted_call: untrusted:read_inbox (chapter 32);
• the irreversible warning, for a tool marked irreversible (chapter 34);
• the answers it takes.

The answers¶

Every pause a run makes, and the answers each takes, generated from the product:

approve runs the call with exactly the arguments shown, and nothing else. A confirmation approves only the call shown: its token covers that tool with those arguments at that point of the run, so a call whose arguments changed after it was shown -- by the model, or by anything else -- is checked again and asks again (chapter 32). skip and cancel_all are in chapter 34.

In Python, reply.waiting lists what waits, and desk.confirm(user, "approve") answers it.

Resuming on any worker¶

A run that waits is kept where storage: says -- sqlite:///runs.db for one machine, postgres://... for several. Every pause has a stable id: the same call, paused for the same reason, has the same id on every worker. So the answer can arrive anywhere: the worker that receives it reads the run from storage, takes the answer to it, and carries on. Nothing is held in the process between a question and its answer.

Several pending at once¶

The agents agent.yaml describes take the calls of a turn in order, each to its end, so a run of theirs waits on one thing at a time. Steps a team writes itself can run calls side by side, and then a run waits on several at once. Each is answered by its own id -- desk.confirm(user, "approve", waiting=id), or the id in POST /threads/{id}/resume -- and the others stay open. With no id, desk.confirm answers the first that waits; GET /threads/{id}/interrupts lists them all.

A stale resume¶

An answer to a pause that is no longer open -- someone else answered it, and the run moved on -- is refused as 409 interrupt_stale, and changes nothing. Two answers to the same pause that arrive together, on one worker or two, are told apart by a claim kept in the run store: the first takes it and resumes the run; the other is interrupt_stale. An answer that carries a token other than the one the person was shown is interrupt_stale too. The interface reads what waits again when it gets one.

Approval from another channel¶

A confirmation can be answered from somewhere other than the chat: a queue, an e-mail, a chat tool your team already uses. The application stalwart serve runs, make_app in stalwart_agents.serve, takes two arguments for it:

• on_interrupt=, told of each new pause as (thread, interrupt). The package's Webhook(url, secret) is one: it posts the pause as JSON, signed in a Stalwart-Signature: t=<time>,v1=<hmac> header, the HMAC-SHA256 of the time and the body under your secret. A receiver accepts a delivery only within five minutes of its own clock, so a captured one cannot be replayed later. A post that fails never fails the turn, which already ran: the answer lists what it could not post under not_notified;
• secret=: where set, a resume must carry a valid Stalwart-Signature too, so only the channel that holds the secret can answer.

stalwart serve itself takes neither; a team that wants approvals from another channel mounts make_app in its own server.

What stalwart serve gives you¶

$ stalwart serve agent.yaml

serves your agents over HTTP, with the interface at /, before you write any interface yourself. Its routes, generated from the package:

Its options:

The operator's rollback undoes a thread's run from outside it. It takes the operator token as Authorization: Bearer <token>, compared in constant time; with no token set, the route is off and answers 403, saying how to set one.

@stalwart-agents/agent-ui¶

The interface stalwart serve shows is a package of its own, for a team that puts the conversation in its own pages:

$ npm install @stalwart-agents/agent-ui

Embedding it¶

Importing the package defines its elements; one of them is the whole conversation:

<stalwart-chat thread="u-17" base="/agent"></stalwart-chat>

base is where your stalwart serve (or your own mount of it) answers; thread is the conversation's id. Every element, generated from the package's source:

<stalwart-confirm> dispatches stalwart-decision with the pause and the answer chosen; <stalwart-chat> sends it to the backend for you.

The backend it talks to¶

The interface talks to base and nothing else: the routes above -- a turn, what waits, a decision, the step stream, a thumb. No telemetry, no font, no script from any other host. Each request names the page's session, once the server has handed one out, so a thumb is taken only from the session that was shown the reply.

Theming it¶

Every colour and size is a CSS custom property, with a light and a dark default; theme="light" or theme="dark" fixes one, and with neither the person's own setting chooses:

Each region is a ::part (listed above), and <stalwart-chat> has header and footer slots for your own content. For the shop's colours, for instance: stalwart-chat { --stalwart-accent: #2e7d32; --stalwart-radius: 4px; --stalwart-font: Georgia, serif; }.

Replacing its words¶

Every word the interface shows is a string with a key, and setStrings({ approve: "Aprovar", skip: "Pular este passo" }) replaces any of them; a {name} in a string is filled where it is shown. The strings as shipped:

The limits¶

• A confirmation approves only the call shown, with its arguments; a changed argument asks again. It proves that a person approved, not that they read it.
• An answer reaches a run that waits; it never changes one that moved on (interrupt_stale).
• A webhook's receiver is yours: the signature proves the delivery came from your server within five minutes, and nothing about who answered it on the other side.
• The interface shows what the run reports -- a refusal's rule, a rollback's undos -- never a reading of the reply's words.

42 · Replies true to what was done¶

After your agent acts, its model writes the reply the person reads, and a model can say a refund went through when it was refused, or name an amount nobody charged. With replies: set, every reply the model writes is held to what was actually done before it is shown: each sentence about an action must agree with that action's outcome, as the run recorded it, and a sentence that does not is replaced by a plain one that does.

The outcomes¶

The run records an outcome for every call it attempted, from the check's answer and what became of the call, never from what the model says. Each outcome, what a sentence about it may say, and the phrasing a sentence that says more is replaced by, generated from the runtime's check:

{action} is the tool's name in words (refund card), {with} the arguments the call was sent, and {because} the reason, where there is one.

The check, step by step¶

1. The reply is read sentence by sentence, with the same reader that reads the person's turns, so I've sent your money back is read as speaking of refund_card.
2. Each sentence is held to the outcome of the action it speaks of: this turn's latest call of that tool, or the session's. A sentence saying a refused refund was made is replaced by the refused phrasing.
3. An action mentioned with no call behind it: a sentence saying a write was done, where nothing in the session attempted it, is replaced by the not_attempted phrasing.
4. Every value a sentence states -- an amount with its currency, a date, an e-mail address, an identifier -- must be one the call was sent or returned. A sentence stating another is replaced by the outcome's phrasing.
5. A write this turn made that the reply leaves out is added, in its outcome's phrasing.
6. The reply rules apply last: personal data, phrases never to say, offers, required sentences (chapter 43).

The reply the person reads is the one the check returns. A replaced sentence is never shown, and each replacement is counted, with no text (chapter 43).

The shop's refunds¶

The shop's desk, with on_unread: model, so that a turn nothing reads goes to its model. The model here is a plain function that stands in for yours: for each turn, the call it asks for, and the reply it then writes:

# Stands in for your model: for each turn, the call it asks for and the reply it then writes.
SCRIPT = {
    "my parcel is lost": (None, ""),
    "the parcel for 4412 is lost": (
        {"name": "get_order", "arguments": {"order_id": "4412"}},
        "Order 4412 has been refunded."),
    "the parcel for 4412 never turned up": (
        {"name": "refund_card", "arguments": {"order_id": "4412", "amount": "all"}},
        "I've refunded order 4412."),
    "the parcel for 4412 went missing": (
        {"name": "refund_card", "arguments": {"order_id": "4412", "amount": 12.50}},
        "I've refunded 15.00 pounds to your card for order 4412."),
    "the parcel for 4412 vanished": (
        {"name": "refund_card", "arguments": {"order_id": "4412", "amount": 12.50}},
        "I've refunded 12.50 pounds to your card for order 4412."),
}


def model(conversation):
    turn = [m for m in conversation if m["role"] == "user"][-1]["content"]
    call, reply = SCRIPT[turn]
    if conversation[-1]["role"] == "user" and call:
        return {"content": "", "tool_calls": [call]}
    return reply

desk = Agent("desk", tools=[get_order, refund_card], model=model, on_unread="model",
             replies={"phrasings": "phrasings.yaml"})


def turn(text, answer="approve"):
    reply = desk.chat(text, user="u-17")
    if reply.waiting:                                   # a write waits for the person
        print(f"{text} -> waits: {reply.waiting[0].wording}")
        reply = desk.confirm("u-17", answer)
        print(f"  {answer} -> {reply.text}")
    else:
        print(f"{text} -> {reply.text}")


turn("my parcel is lost")                    # nothing reads it: asked once to say it again
turn("the parcel for 4412 is lost")          # the model says a refund was made; none was called
turn("the parcel for 4412 never turned up")  # the refund was refused; the model says it was made
turn("the parcel for 4412 went missing")     # done, but the model states another amount
turn("the parcel for 4412 vanished")         # done, and the reply says what was done
turn("the parcel for 4412 went missing", answer="skip")   # the person said no: not attempted

my parcel is lost -> I didn't recognise anything in that. I can get order or refund card. Which do you mean?
the parcel for 4412 is lost -> Not done: refund card.
the parcel for 4412 never turned up -> I couldn't refund order 4412: argument_schema.
the parcel for 4412 went missing -> waits: Run refund_card?
  approve -> Refunded 12.5 pounds for order 4412.
the parcel for 4412 vanished -> waits: Run refund_card?
  approve -> I've refunded 12.50 pounds to your card for order 4412.
the parcel for 4412 went missing -> waits: Run refund_card?
  skip -> Not done: refund card.

Line by line:

• the first turn is one nothing reads, so the person is asked once to say it again; the next goes to the model;
• the model said order 4412 has been refunded after reading it, and no refund was called: Not done: refund card.;
• the refund the model asked for was refused, its amount not a number: the shop's own refused phrasing;
• the refund ran with 12.50, and the model wrote 15.00: the sentence is replaced by the shop's done phrasing, with the amount the call was sent;
• the refund ran, and the model's sentence says what ran: it is shown as written;
• the person said no: Not done: refund card.

Outcome phrasings¶

A phrasing is what a reply says about a write for one outcome. The shop's, for refund_card:

# How a reply tells each write's outcome, in the shop's words. An outcome not listed takes the default.
refund_card:
  done: "Refunded {argument.amount} pounds for order {argument.order_id}."
  refused: "I couldn't refund order {argument.order_id}{because}."

An outcome a tool has no phrasing for takes the default above, in the language replies: {language: pt} names (en by default). A phrasing may use {action}, {Action} (capitalised), {with}, {because}, {reason}, {argument.NAME} and {result.NAME}. Your organisation's phrasings are also kept on the console's Phrasings page and at /v1/outcome-phrasings, in versions.

The template composer¶

With replies: {composer: template}, the reply is the phrasings themselves: whatever the model writes, the person reads the phrasing of each call this turn made, in order:

template = Agent("desk", tools=[get_order, refund_card], model=model, on_unread="model",
                 replies={"phrasings": "phrasings.yaml", "composer": "template"})
for text in ("my parcel is lost", "the parcel for 4412 vanished"):
    reply = template.chat(text, user="u-18")
    if reply.waiting:
        reply = template.confirm("u-18", "approve")
    print(f"{text} -> {reply.text}")

my parcel is lost -> I didn't recognise anything in that. I can get order or refund card. Which do you mean?
the parcel for 4412 vanished -> Refunded 12.5 pounds for order 4412.

With the template composer, no reply the person reads is written by a model (chapter 45).

Where it runs¶

The check runs in the runtime on your machine, on every plan, with no network. The reply, the arguments and the results stay on your machine: only the counts are reported, with your licence's other meters.

The limits¶

• The check is deterministic and bounded, and it does not prove a reply true. It removes what the run's records do not support: a sentence about an action that disagrees with its outcome, and a value the action was neither sent nor returned.
• It reads the sentences the reader can tie to a tool. A sentence it ties to no tool is held only to the reply rules (chapter 43), and a value is checked only where it has the shape of one: an amount with its currency, a date, an address, an identifier.
• The outcome is the run's record, which for a call that left your machine is what your tool returned: a tool that reports success for something that failed is reported as done.
• It holds the replies your model writes. A reply a person reads that no model wrote -- a fixed question, a refusal, a phrasing -- is the product's own wording.

43 · Reply safety¶

With replies: {rules: ...} set, every turn a person sends is screened for personal data before the router or the model reads it, and every reply the model writes is screened again before it is shown, whether or not an action came before it. Your rules say what each class of personal data gets, which phrases are never said, which offers a reply may make and which sentences must go with a write. No model judges any of it: the runtime on your machine decides each rule.

Personal data¶

The classes the runtime finds, how each is found, what it gets where your rules say nothing, and how it is masked, generated from the runtime's detectors:

A card number is found by its checksum, not by its shape: sixteen digits that fail the Luhn check are left as they are, because they are not a card number. Each class takes one of three actions:

A value an outcome sentence may repeat. A sentence reporting a write may name a value the write was sent -- the card a refund went to -- and that value is masked in it rather than removed, whatever the class's action, unless the class is allowed.

Wording¶

• never_say: phrases never said. A sentence holding one, whole words and any case, is removed.
• offers: the offers a reply may make. A sentence holding one of the markers (such as discount or voucher) and none of the allowed offers is removed.
• must_accompany: sentences that must go with something: a disclaimer added when a tool's call ran in this turn (when_done), or when an answer cites a passage with a tag (when_cited). It is added once, where the reply does not already say it.

The reply rules file¶

The rules are one file, YAML or JSON, named in agent.yaml as replies: {rules: reply-rules.yaml} or in Python as Agent(..., replies={"rules": "reply-rules.yaml"}). The shop's:

# The shop's reply rules, as the console's Reply rules page saves them.
personal_data:
  card: redact            # the default, written out
  national_id: withhold   # a turn holding one reaches no model; a reply holding one loses the sentence
  email: allow
never_say:
  - "guaranteed refund"
must_accompany:
  - when_done: refund_card
    add: "Refunds reach the card in three to five working days."

Every key the runtime reads, generated from the runtime:

The same rules are kept on the console's Reply rules page, as a form over the same file, and at /v1/reply-rules, in versions. A file the runtime cannot read -- a class it does not know, an action that is not one of the three -- is refused with the key that is wrong.

The shop's replies, screened¶

The model below stands in for yours, and records what it was given:

SEEN = []          # what the model was given, turn by turn

# Stands in for your model: what it is asked, and the reply it writes.
SCRIPT = {
    "zork": (None, ""),
    "my card is **** 1111, is it on file?": (
        None,
        "Your card 4111 1111 1111 1111 is on file; the card 4111 1111 1111 1112 is not."),
    "the parcel for 4412 vanished": (
        {"name": "refund_card", "arguments": {"order_id": "4412", "amount": 12.50}},
        "I've refunded 12.50 pounds for order 4412. It is a guaranteed refund."),
}


def model(conversation):
    turn = [m for m in conversation if m["role"] == "user"][-1]["content"]
    SEEN.append(turn)
    call, reply = SCRIPT[turn]
    if conversation[-1]["role"] == "user" and call:
        return {"content": "", "tool_calls": [call]}
    return reply

desk = Agent("desk", tools=[get_order, refund_card], model=model, on_unread="model",
             replies={"rules": "reply-rules.yaml"})

for text in ("zork",
             "my card is 4111 1111 1111 1111, is it on file?",
             "my social security number is 123-45-6789",
             "the parcel for 4412 vanished"):
    reply = desk.chat(text, user="u-17")
    if reply.waiting:
        reply = desk.confirm("u-17", "approve")
    print(f"{text}\n  -> {reply.text}")

print("The model was given:", SEEN)

zork
  -> I didn't recognise anything in that. I can get order or refund card. Which do you mean?
my card is 4111 1111 1111 1111, is it on file?
  -> Your card **** 1111 is on file; the card 4111 1111 1111 1112 is not.
my social security number is 123-45-6789
  -> That message held details I can't take here, so I haven't read it or done anything. Please send it again without them.
the parcel for 4412 vanished
  -> I've refunded 12.50 pounds for order 4412. Refunds reach the card in three to five working days.
The model was given: ['my card is **** 1111, is it on file?', 'the parcel for 4412 vanished', 'the parcel for 4412 vanished']

• The card number the person typed was masked before anything read the turn: the model was given **** 1111.
• The model's reply named the same valid card, masked, and a sixteen-digit number that fails the checksum, left as it is.
• The turn with a social security number was held back whole, under national_id: withhold: nothing read it, and the person was asked to send it again without it. It is not in what the model was given.
• After the refund ran, It is a guaranteed refund. was removed (never_say), and the disclaimer the rules require for a refund was added.

The counts¶

Each check is counted, by the meter it is recorded under, with your licence's other meters. A count is a number: it carries no text, no value and no turn.

The limits¶

• A detector finds the classes it names, and no others. A name, a postal address, a date of birth or an identifier of another country is not detected, and is left as it is.
• A detector finds a value written as the detector reads it. A card number spelt out in words, or split across two messages, is not found.
• No model judges a reply. A rule matches the words it is given: a phrase said in other words is not a phrase never_say lists.
• The screen holds the turns the agent reads and the replies its model writes. A reply no model wrote -- a fixed question, a refusal, a phrasing -- is the product's own wording.

44 · Grounded answers from your sources¶

Some turns only ask for information -- how long does a refund take? -- and no tool answers them. With on_unread: answer, such a question is answered from your own documents: the passages that match it are found on your machine, the answer is made of what they say, and every sentence carries the passage it came from. A sentence a passage does not support is removed, and where nothing is left the agent says it has no answer.

Your documents, indexed on your machine¶

stalwart sources build reads your documents and writes an index, here, with the licensed runtime:

$ stalwart sources build docs/delivery.md docs/refunds.md --out sources.json
Indexed 2 document(s), 3 passage(s), into sources.json (ba9e751e249e).
$ echo $?
0
$ stalwart sources ask "How long does a refund take?" --index sources.json
A refund reaches the card the order was paid with in three to five working days.  [refunds.md > How long a refund takes]
$ echo $?
0

• Markdown (.md) and HTML (.html, .htm) are split at their headings and then at blank lines: each paragraph is a passage, under the heading above it. Plain text is split at blank lines. A PDF's text is read where pypdf is installed.
• Each passage is indexed by its words and its heading's words; a vocabulary file (--vocabulary) adds the terms and kin your vocabulary declares, so a question in your customers' words finds a passage in yours.
• Re-indexing is building again: the index is a file, with its digest printed, and the agent reads the file it is given.
• A file that cannot be read -- missing, empty, not UTF-8, a PDF with no pypdf -- stops the build with its name and exit status 2.

stalwart sources ask answers a question from the index as the agent would with the extract composer, each sentence with the document and heading it came from, so you see what your users will be told.

The documents and the index stay on your machine. Stalwart's servers never receive them.

Answering in the agent¶

An agent answers from the index when its on_unread is answer -- the third choice beside refuse and handing the turn on (chapter 45) -- and answers: names the index:

desk = Agent("desk", tools=[get_order, charge_card], model="anthropic:claude-sonnet-5-5",
             on_unread="answer", answers={"index": "sources.json", "composer": "extract"})

for text in ("How long does a refund take?",
             "Can a gift card be refunded?",
             "charge order 4411 40.00"):
    reply = desk.chat(text, user="u-17")
    shown = reply.waiting[0].wording if reply.waiting else reply.text
    print(f"{text}\n  -> {shown}")

How long does a refund take?
  -> A refund reaches the card the order was paid with in three to five working days.
Can a gift card be refunded?
  -> An order can be refunded within 30 days of delivery. A gift card cannot be refunded.
charge order 4411 40.00
  -> Run charge_card?

The two questions were answered from the passages; charge order 4411 40.00 asks for something to be done, so it went to the charge and its confirmation, and never to the documents. An action is never answered with documentation: a passage describing how a refund works is not a refund. A question asked beside an action -- refund order 4411 and tell me how long it takes -- is answered after the action has run.

answers: takes index, k (at most this many passages, 3 by default), floor (no passage scoring below it, 0.3 by default, on a scale of 0 to 1) and composer. Those an agent does not give come from your console's Settings page, at the conversation's first turn.

The composers¶

• extract: the answer is the passages' own sentences that share a term with the question, in rank order. No model is called, and every sentence is supported by construction.
• compose: your agent's own model writes the answer from the numbered passages, with your key, marking each sentence with the passages it draws on. Each model call is counted as one answer_calls. An answer left with no sentence after the check gets one second attempt.

Which plans include each, generated from the plans' rows:

Either way, the runtime checks every sentence against the passages it cites: each number, date, amount, name and negation in it must be in one of them. A sentence that fails is removed. Retrieval and the check run in the runtime, with no network call; compose calls your model, and nothing else.

Citations¶

Every sentence of an answer is bound to its passages: the document, the heading and, where the document gave one, its link. stalwart sources ask prints them, as above, and stalwart serve's interface shows them under the answer (chapter 35). A rule of chapter 43 can add a sentence whenever a passage with a given tag is cited.

No grounded answer¶

When no passage scores above the floor, or no sentence survives the check, the outcome is no_grounded_answer, and the agent goes on as if on_unread were refuse: the person is asked once to say it again, and then told it cannot be done. Here the refunds document is deleted and the index built again:

$ rm docs/refunds.md
$ stalwart sources build docs/delivery.md --out sources.json
Indexed 1 document(s), 1 passage(s), into sources.json (1e1f014fea6f).
$ echo $?
0
$ stalwart sources ask "How long does a refund take?" --index sources.json
No grounded answer: no passage scores above the floor. The agent would refuse or hand the turn off.
$ echo $?
1

The command exits 1 for no grounded answer.

What is kept, and where¶

Nothing of it is kept by us. The documents, the index, the questions and the answers are on your machine and in your storage. Only the count of answer_calls is reported, with your licence's other meters.

The limits¶

• A supported sentence agrees with its passage. It does not make the passage true, or current: a document that is out of date gives answers that are out of date.
• Retrieval matches words, weighed by how rare they are in your documents. A passage that shares enough of a question's words to score above the floor is used, whether or not it is about the question; the floor and k are how strict it is.
• The check holds numbers, dates, amounts, names and negations to the passages. A sentence that rewords a passage and changes its meaning in other words can pass it.

36 · Seeing what happened¶

When a run does something you did not expect, you want to see it step by step: what the router read, what each check decided and why, which call ran, what was undone. stalwart trace shows a run as a timeline, from the storage your agent already keeps or from the spans a tracer recorded, and every decision on it names the rule that made it.

A run as a timeline¶

stalwart trace has three subcommands:

• stalwart trace show: the timeline in the terminal, one block per step, each check with its answer and its reasons, each pause for a person, each undo and the step it undoes.
• stalwart trace html ... -o run.html: the same timeline as one self-contained HTML file, with no script and no request to anywhere, for a ticket or an audit.
• stalwart trace diff a.jsonl b.jsonl: where two runs of the same input went different ways: the first step whose decisions differ, and what only each run did there.

show and html read either a spans file or, with --thread and --store, your agent's storage.

The same from storage alone¶

A run your agent kept in storage: can be read back with nothing recorded beforehand. E6's failed order (chapter 34), with the shop's storage on disk:

desk = Agent.from_file("agent.yaml")          # storage: sqlite:///runs.db

reply = desk.chat("charge order 4411 40.00, then update the crm for order 4411, "
                  "then provision the account for order 4411", user="u-17")
while reply.waiting:
    reply = desk.confirm("u-17", "approve")   # each write approved as it comes
print(reply.text)
desk.close()

This run stopped: provision_account failed (RuntimeError). Undone, latest first: update_crm (by restore_crm), charge_card (by refund_card).

The conversation is desk/u-17: the agent's name and the person's. Read back from the storage:

$ stalwart trace show --thread desk/u-17 --store sqlite:///runs.db
Run: thread desk/u-17
step -1
  node      input
step 0
  node      run
step 1
  node      run
step 2
  node      run
  check     charge_card -> confirm
  interrupt charge_card [write] (confirm)
step 3
  node      run
  check     charge_card -> allow
step 4
  node      run
  call      charge_card
step 5
  node      run
step 6
  node      run
step 7
  node      run
  check     update_crm -> confirm
  interrupt update_crm [write] (confirm)
step 8
  node      run
  check     update_crm -> allow
step 9
  node      run
  call      update_crm
step 10
  node      run
step 11
  node      run
step 12
  node      run
  check     provision_account -> confirm
  interrupt provision_account [write] (confirm)
step 13
  node      run
  check     provision_account -> allow
step 14
  node      run
  call      provision_account
step 15
  node      run
  undo      restore_crm -> allow undoes step 3
step 16
  node      run
step 17
  node      run
  undo      refund_card -> allow undoes step 1
step 18
  node      run
step 19
  node      run
step 20
  node      run

Each write was checked and asked about (confirm, [write]), approved (allow), and run; provision_account failed at step 14, and the two undos follow, each checked, latest first: restore_crm undoes the CRM update (history step 3) and refund_card the charge (history step 1). A step with nothing under it is one that only moved the run on.

Spans for any OpenTelemetry tool¶

A tracer records one span around each of these, generated from the package's tracer:

A tool check's span carries its decision, as these attributes and as a stalwart.decision event:

gen_ai.* are OpenTelemetry's GenAI conventions, so any tool that reads them shows the calls; stalwart.* are ours. With pip install "stalwart-agents[otel]", the spans go to OpenTelemetry's tracer provider, and from it to whichever exporter you configured. With no exporter at all, a JSON-lines tracer writes them to a file, which stalwart trace show and diff read.

Content is off by default¶

No argument value, message, result or turn is put on a span unless the tracer is made with capture_content=True. Where content is off, a 16-character digest stands in for each, so two spans can be compared without either holding the value. Turning it on adds exactly these attributes: