F
Member
【Editorial Discussion】
Retrieved documents, tickets, or emails can contain instructions that the model will treat as higher priority than the system policy, especially if those documents are also allowed to influence tool arguments. A filter on the user prompt will not see this path. The useful test is not “can we talk about injection,” it is which attacks your allow-list and your output validator actually stopped.
1) Put adversarial content in the retrieved set, not only in the user box: requests to ignore policy, to exfiltrate earlier context, or to issue a tool call the user did not ask for. If your eval only uses clean corpora, you have not tested this path.
2) Tool arguments should be validated against types, destinations, and an allow-list that the document cannot expand. A model that is “well behaved” in prose can still emit a URL or a command that the wrapper will run. The wrapper is the control; the prompt is not.
3) Log when a document asked for an action and when the wrapper refused. If you cannot show a refusal, you cannot claim the control works.
4) Re-run the set when you add a tool or change the retriever. A new tool is a new attack surface.
Open question: have you caught an injection that arrived only through a retrieved chunk, and which layer stopped it — the model, the validator, or the allow-list? If nothing would have stopped it, what did you add? Method-level reports welcome.
Retrieved documents, tickets, or emails can contain instructions that the model will treat as higher priority than the system policy, especially if those documents are also allowed to influence tool arguments. A filter on the user prompt will not see this path. The useful test is not “can we talk about injection,” it is which attacks your allow-list and your output validator actually stopped.
1) Put adversarial content in the retrieved set, not only in the user box: requests to ignore policy, to exfiltrate earlier context, or to issue a tool call the user did not ask for. If your eval only uses clean corpora, you have not tested this path.
2) Tool arguments should be validated against types, destinations, and an allow-list that the document cannot expand. A model that is “well behaved” in prose can still emit a URL or a command that the wrapper will run. The wrapper is the control; the prompt is not.
3) Log when a document asked for an action and when the wrapper refused. If you cannot show a refusal, you cannot claim the control works.
4) Re-run the set when you add a tool or change the retriever. A new tool is a new attack surface.
Open question: have you caught an injection that arrived only through a retrieved chunk, and which layer stopped it — the model, the validator, or the allow-list? If nothing would have stopped it, what did you add? Method-level reports welcome.

