Testing for prompt injection is easy to do badly and produce a false sense of confidence. A tester types a few well-known adversarial phrases into a chat interface, the model resists them, and the report concludes the system is resistant to prompt injection. This misses most of what actually matters: the indirect paths an attacker is more likely to use, and the question of whether a successful injection would actually cause harm even if it does get through.
Map every content path before writing a single test
Before testing anything, list every source of text the model processes: direct user input, retrieved documents, fetched web pages, email or message content, tool responses, and any memory the system reads back. Each of these is a separate attack surface with its own realistic delivery mechanism. A test plan that only covers direct input has tested one of potentially six or more distinct paths.
Test direct injection with a range of techniques, not just the obvious ones
For the direct path, go beyond the most well-known phrasing, "ignore your previous instructions," which most systems have specifically been tuned to resist. Test role-play framings, requests disguised as translation or debugging tasks, multi-turn setups that build context across several messages before the actual injection attempt, and encoded or obfuscated payloads designed to evade simple keyword-based filtering. Published research on injection techniques is a useful and constantly updated source of realistic attack patterns to test against.
Test indirect injection by planting content in what the model will process
For retrieval, create a test document containing an injection attempt and confirm it gets indexed and eventually retrieved for a relevant query, then observe whether the model follows the planted instruction. For browsing tools, set up a test page with hidden or subtly placed injection content and have the agent fetch it. For email or messaging assistants, send a test message crafted to look like an ordinary inquiry while carrying an injection attempt. For tool responses, if feasible, simulate a compromised or manipulated response from an integrated service and observe how the agent's next step is affected.
This requires more setup than direct testing, and it is the category most likely to be skipped under time pressure, which is exactly why it deserves deliberate planning rather than being left as an afterthought.
Measure the actual consequence, not just whether the model complied
The most important shift in a mature test process is measuring what happens as a result of a successful injection, not just whether the model's response text changed. If an injection causes the model to say something it should not, that is worth noting. If an injection causes the model to attempt a tool call it should not be authorized to make, that is a more serious finding, and the test should also check whether the downstream authorization or policy check actually stopped that attempt. A successful injection that fails to produce any real consequence, because an independent enforcement layer caught it, is a very different finding than one that results in an actual unauthorized action.
Test across model versions and configuration changes
An injection technique that fails against one model version or one prompt configuration may succeed against another, and a model provider updating their model can silently change results that were previously stable. Re-running a core set of injection tests after any model version change, prompt structure change, or new tool integration catches regressions that a one-time test would miss entirely.
Test multi-turn scenarios, not just single messages
Some injection techniques build across several conversational turns, establishing a seemingly innocuous premise early and introducing the actual manipulative instruction several messages later, once the model's context already contains supporting setup. Testing only single-message injection attempts misses this entire category, which tends to be more effective against systems specifically hardened against single-message attacks.
Document findings by path and by consequence
A useful test report separates findings by content path, direct, retrieval, browsing, email, tool response, memory, and by consequence, text-only deviation versus an actual unauthorized action attempt. This makes it clear to engineering teams exactly which paths need attention and which findings are urgent because they resulted in a real action versus ones that are lower priority because an existing enforcement layer already contained them.
Where TELEON fits
Where TELEON's policy enforcement is deployed through the gateway or supported middleware, injection testing can specifically verify whether an attempted unauthorized tool call was actually blocked at that layer, turning "the model was tricked" into a testable question about whether the enforcement point held, which is the more actionable finding for a security team to act on.
The design and execution of the injection tests themselves, across every content path and technique category, remains work that has to be done by the team testing the application, since it depends on the application's specific tools, integrations, and content sources.
A short testing checklist
- Map every content path the model processes before designing tests.
- Test direct injection with a wide range of techniques, updated
regularly.
- Test indirect injection by planting content in documents, web pages,
email, and tool responses.
- Measure actual consequence, especially attempted unauthorized actions,
not just response text.
- Re-test after model version changes and new tool integrations.
- Include multi-turn scenarios, not only single-message attempts.
- Document findings by content path and by consequence severity.
Testing for prompt injection well requires covering every path content can enter the model, and measuring whether an enforcement layer actually stopped the consequence, not just whether the model's wording changed. A narrow test that only covers the chat box will always look more reassuring than the system actually is.