Semantic Cache Poisoning: The LLM Shortcut Attack Nobody Is Monitoring

A semantic cache looks like free performance. Embed the incoming request, find a sufficiently similar request, and return the answer you already paid to generate. Latency falls. Token spend drops. Everyone gets to pretend the architecture team found money under the couch.

Then one bad answer becomes a reusable asset.

That is the core risk of semantic cache poisoning. A malicious, compromised, or merely incorrect response enters the cache. Future requests do not need to repeat the original exploit. They only need to land close enough in embedding space to retrieve the poisoned result.

Why This Is Worse Than Ordinary Cache Poisoning

A conventional web cache usually keys on concrete values: URL, headers, method, and sometimes cookies. A semantic cache keys on approximate meaning. Two strings can differ completely and still map to nearby vectors.

That fuzziness is the product feature—and the security problem.

Suppose a support agent answers questions using tenant-specific documents. An attacker crafts a request that causes the model to ignore retrieved policy and output a bogus refund procedure. If the final response is cached under a broad semantic neighborhood, another customer’s innocent “How do I get a refund?” query may retrieve it directly. The model never runs again. Your prompt-injection defenses do not get a vote.

The blast radius expands when the cache ignores identity and context. These dimensions must be part of the effective key:

• tenant and user authorization scope
• model and model-version identifier
• system-prompt and policy version
• retrieval corpus and document version
• tool availability and permission set
• locale, jurisdiction, and product plan
• safety-filter and output-schema version

If your cache key is just an embedding plus a similarity threshold, it is not a cache. It is a cross-context answer lottery.

Four Ways Poison Enters the Cache

The obvious path is prompt injection. A user supplies content that changes the model’s behavior, and the resulting answer is stored. But production failures rarely stay obvious.

Compromised retrieval content can poison a response without a suspicious user message. A malicious document enters a knowledge base, gets retrieved, and convinces the model to produce an unsafe answer. The cache preserves the result even after the document is removed.

Authorization mistakes can turn a correct answer into poisoned shared state. The response may be valid for an administrator but dangerous when replayed to a standard user.

Stale policy is quieter. A cached answer approved under yesterday’s rules survives a policy update. Now the system returns forbidden content without calling the new guardrail chain.

Evaluation contamination happens when test or red-team traffic writes to production cache namespaces. Your own jailbreak suite can become the seed corpus for real users. Spectacular work.

Build a Cache Admission Gate

Do not cache every successful 200 response. Cache admission should be an explicit security decision.

First, classify the route. High-risk tasks involving money movement, account recovery, legal conclusions, medical advice, or destructive tool calls should usually bypass semantic response caching. Cache retrieved evidence if appropriate; do not blindly cache the final decision.

Second, require provenance. Store the model version, prompt hash, policy version, retrieval document IDs, authorization scope, and guardrail result beside the answer. An entry without provenance is unverifiable sludge.

Third, validate the output before admission. Schema validation is the floor. Add policy checks, citation checks, tool-result reconciliation, and route-specific invariants. If an answer says a refund was issued, the transaction system—not the model prose—must prove it.

Fourth, separate cache namespaces aggressively. Tenants should not share response entries unless the content is intentionally public and context-free. Admin and user permission scopes should never collide.

Fifth, set short time-to-live values where knowledge or policy changes quickly. Invalidate by policy version and document version, not only by clock time.

Retrieval Needs More Than a Similarity Score

Similarity is not correctness. Add lexical constraints for critical entities, structured filters for product and locale, and a minimum confidence margin between the best and second-best candidates. Ambiguous neighborhoods should fall through to fresh generation.

Keep the cached request and answer available for inspection. When a match occurs, log:

• source entry ID
• similarity score and threshold
• identity and authorization namespace
• age of the entry
• original provenance
• downstream validation result

Monitor hit rates by route, but also monitor rejection rates, safety-check disagreements, and sudden clusters of queries mapping to one entry. A cache entry that starts serving an unusual share of traffic deserves quarantine.

Test the Attack You Actually Built

Your red team should test persistence, not just one-turn jailbreaks.

Seed malicious documents, send adversarial requests, and verify that unsafe responses never enter the cache. Then remove the trigger and send ordinary paraphrases. If any retrieve the poisoned output, the exploit is successful.

Test cross-tenant collisions, policy upgrades, model rollbacks, and test-environment leakage. Verify that cache invalidation removes both the entry and any derived indexes. Run these tests whenever embedding models, chunking, thresholds, or prompt policy changes.

Semantic caching is useful. It is also a new execution path with its own authorization, validation, and observability requirements. Treating it as a transparent performance layer is how one compromised response becomes everybody’s answer.