A secret, an API key, a database password, an internal service token, should never appear in an AI model's context window. It happens regularly anyway, usually not through some dramatic breach but through ordinary development shortcuts nobody flagged as a security concern at the time. Knowing the specific paths this happens through makes it a lot easier to catch and prevent, rather than treating it as a vague risk managed with a general reminder to "be careful."
Copied configuration during prompt development
The most common path: a developer building or testing a prompt template copies a real configuration value, an API key, a connection string, a service credential, into the prompt for convenience while developing, and never gets around to removing it before the template drifts toward production. This happens because prompt development often feels more like writing documentation than writing code, so it doesn't automatically trigger the same secret-scanning discipline a code commit would. Treating prompt templates with the same secret-scanning rigor as source code closes this specific path outright.
Secrets embedded in documents that feed retrieval
For RAG systems, a source document that happens to contain a real credential, an internal wiki page with a database connection string thrown in as an example, a troubleshooting document referencing a real API key used for testing, can carry that secret straight into a model's context the moment the document gets retrieved for a relevant query. This path is easy to miss because the secret was never placed there with any AI system in mind. It was just sitting in a document that later became a retrieval source, with nobody re-reviewing its content specifically for this risk.
Tool responses that echo back credentials
If a tool an agent calls returns a response including a credential, maybe because the tool wraps an API that hands back a token as part of its response payload, that credential can land in the model's context if the full tool response gets passed back unfiltered. This deserves specific attention when integrating any tool touching authentication or configuration systems, since the tool's response format was often designed for a completely different consumer, a script, another service, that would have handled a returned credential very differently than dumping it straight into a model's context.
Why prompt-level instructions do not solve this
Telling a model not to repeat or act on any secrets it encounters doesn't stop the secret from having entered the context in the first place, and it doesn't reliably stop the model from including it in a response if a user, through ordinary conversation or a manipulation attempt, asks a question that leads the model right back to it. The only reliable fix is keeping the secret out of the context entirely, not asking the model to handle it carefully once it's already sitting there.
Secret scanning needs to cover AI-specific content paths
Standard secret-scanning tools built for source code repositories don't automatically cover prompt templates, document sources feeding retrieval, or logged conversation content, not unless someone specifically configures them to scan those locations too. Extending secret-scanning coverage to these AI-specific content paths, prompt template files, ingested document sources before they're indexed, logs capturing prompt or response content, closes a gap a code-focused scanning setup would otherwise miss entirely.
Rotating any secret that may have been exposed this way
If a secret turns up in a model's context, whether through a prompt template, a retrieved document, or a tool response, treat it exactly like any other suspected secret exposure: rotate it, and review everywhere it might have propagated, logs, caches, any downstream system that could have captured a response containing it, rather than assuming the exposure was contained just because it got discovered and the immediate source got fixed.
Where TELEON fits
TELEON's privacy vault tokenization can substitute references for sensitive values before they reach a model's context, which reduces the risk of a secret being included directly, though the more fundamental fix remains keeping secrets out of prompts, documents, and tool responses in the first place through the specific engineering practices described above.
A short checklist
- Apply secret scanning to prompt template files with the same rigor as
source code.
- Review documents feeding retrieval specifically for embedded secrets
before ingestion.
- Filter tool responses to remove credentials before they reach a
model's context.
- Do not rely on prompt-level instructions to handle secrets that have
already entered context.
- Extend secret-scanning coverage to logs capturing prompt and response
content.
- Rotate any secret discovered to have been exposed through an AI
content path, and review its full propagation.
Secrets end up in AI context windows through ordinary development shortcuts and overlooked content paths, not through anything specific to how models work, which means the fix is the same discipline already applied to code, extended deliberately to cover prompts, retrieval sources, and tool integrations.