Rebuilding an adversarial test set from scratch every time a system needs testing wastes effort and produces inconsistent coverage across different testing rounds. A maintained test corpus, a growing, organized collection of injection attempts, edge cases, and past findings, makes repeat testing faster, more consistent, and cumulatively more thorough as it absorbs lessons from every previous testing round instead of starting fresh every time.
Start with known technique categories as the initial seed
Begin the corpus with test cases representing established, published technique categories: direct injection phrasings from known research, indirect injection payloads formatted for documents and web content, common jailbreak framings. This gives the corpus useful initial coverage without the team having to independently rediscover every known technique, and it can get organized by category from the start to stay navigable as it grows.
Add every real finding as a permanent regression test
Every time adversarial testing or a real incident reveals a specific successful manipulation technique against your system, add that case to the corpus as a permanent entry, tagged with what system and version it succeeded against. This turns every discovered gap into ongoing protection: future testing rounds automatically include this case, catching a regression if a code or model change ever reopens a previously fixed vulnerability, which is a common and easy-to-miss failure mode without this kind of accumulated regression coverage.
Organize by content path and technique category
Structure the corpus so test cases are tagged by the content path they target, direct input, retrieval, browsing, email, and by technique category, straightforward injection, multi-turn, jailbreak-adjacent. That organization makes it possible to run a targeted subset relevant to a specific change, running only the retrieval-path cases when a change specifically touches the retrieval pipeline, say, instead of running the entire corpus every time when a faster, targeted subset gets the job done.
Include cases with known expected outcomes, not just adversarial input
For each test case, record not just the adversarial input but the expected outcome: should this case get blocked entirely, should the model resist the manipulation, or is this a case where the manipulation might succeed at the language level but should still get contained by a downstream authorization check. This expected-outcome annotation is what lets the corpus run with automated pass or fail evaluation, rather than needing manual review of every result every single time it gets exercised.
Version the corpus alongside the systems it tests
As the corpus grows and systems evolve, maintain version awareness: which test cases apply to which system or which version, since a case added specifically for one system's particular tool integration might not apply to a different system with entirely different tools. That keeps the corpus useful and navigable, rather than turning into an undifferentiated pile of cases where testers can't easily tell which subset applies to their specific target.
Periodically prune or update stale cases
As models and defenses evolve, some corpus entries can become permanently ineffective, techniques no model or defense configuration in current use is vulnerable to anymore, while staying historically interesting. Periodically review the corpus for cases that no longer provide meaningful signal, and lean toward archiving rather than deleting them, since a technique that seems obsolete can sometimes become relevant again against a different model or configuration down the line.
Share the corpus across teams within the organization
If multiple teams within an organization build separate AI features, a shared, centrally maintained corpus, with team-specific extensions layered on top of the shared core, keeps each team from independently rediscovering the same known techniques and inconsistently covering the same fundamental categories. This ties into the broader governance benefit of shared infrastructure covered elsewhere, just applied to testing assets instead of runtime enforcement.
Automate running the corpus as part of regular testing
Once the corpus reaches a useful size, automate running it, or a relevant subset, as part of continuous integration or a scheduled testing job, rather than relying on manual execution before each release. That turns regression coverage from the accumulated corpus into a routine, low-effort check instead of a manual task competing with other testing priorities for limited time.
Where TELEON fits
Where TELEON's policy enforcement is deployed through the gateway or supported middleware, running the corpus can specifically verify whether cases expected to be contained by downstream authorization actually are, providing the enforcement-level verification that complements the model behavior observation the corpus also captures.
Building and curating the corpus itself, seeding it with known techniques, adding real findings, and organizing it for practical use, remains ongoing work that has to be maintained directly by the team responsible for AI security testing.
A short checklist
- Seed the corpus with known technique categories from published
research.
- Add every real finding as a permanent regression test case.
- Organize entries by content path and technique category for targeted
testing.
- Annotate each case with its expected outcome for automated evaluation.
- Maintain version awareness linking cases to relevant systems.
- Periodically prune or archive stale, no-longer-effective cases.
- Share a common corpus core across teams within the organization.
- Automate running the corpus as part of regular continuous testing.
A maintained AI security test corpus turns every past finding into permanent, reusable protection against regression, and it makes each subsequent round of adversarial testing faster and more consistent than starting from scratch every time.