Retrieval Freshness SLOs: Stop Your RAG System From Serving Yesterday

Your RAG system can retrieve the “right” document and still be wrong because the document is old.

This failure is brutal because it looks healthy. The vector search returns a high score. The model quotes the source. The answer has a citation. Meanwhile, the pricing policy changed yesterday, the security runbook changed an hour ago, or the customer’s account state changed thirty seconds ago.

A retrieval freshness SLO turns that hidden decay into an operating constraint. It defines how quickly source changes must become eligible for retrieval and what the application should do when the pipeline misses that target.

Define Freshness as an End-to-End Delay

Ingestion teams often report that a crawler ran successfully. That is not the same as useful freshness.

Measure from the source change to query eligibility:

source updated
  -> change detected
  -> content fetched
  -> parsed and normalized
  -> chunked and embedded
  -> index committed
  -> query path sees new version

The freshness delay is the time between the first and last event. Each stage needs timestamps and a stable source version. If your logs contain only “job completed,” you cannot tell whether the job processed the latest revision or replayed a week-old snapshot.

Use the source’s native revision ID, ETag, commit SHA, database sequence, or updated timestamp where available. Hash normalized content as a fallback. Store that identity on every chunk and carry it into the retrieval trace.

Set Different Objectives by Risk

One global freshness target is lazy architecture. A company handbook and a live account balance do not age at the same speed.

Classify collections by update rate and consequence:

• product documentation may tolerate hours;
• legal or security policy may require minutes;
• inventory, access, and transaction state may require seconds or a live tool call;
• immutable archives may have no meaningful freshness pressure.

Define a percentile target, not just an average. “Ninety-nine percent of critical policy changes become queryable within ten minutes” is testable. “The index is usually fresh” is wallpaper.

Pair the target with an error budget. A pipeline can miss occasionally, but repeated misses should stop risky releases and trigger investigation. Track lag by tenant, connector, document class, and region because a healthy aggregate can hide one broken source.

Make Staleness Visible at Answer Time

The query path needs to know more than semantic similarity. Every retrieved chunk should expose source version, source update time, ingestion time, and index commit time.

Then define a freshness policy. If a critical document is older than its maximum acceptable age, the system can:

• call a live authoritative API;
• request an on-demand refresh;
• answer with an explicit “as of” time;
• restrict the response to stable information;
• refuse a high-risk action until current state is verified.

Do not silently serve stale content and hope the model hedges. The model cannot infer a policy update that is absent from context.

For user-facing citations, show the source date when it materially changes trust. Internally, preserve the exact version used. When a bad answer is reported, operators should be able to reproduce the evidence set instead of searching whatever happens to be current now.

Test the Pipeline With Synthetic Changes

Freshness requires active testing. Create a harmless canary document for each connector and update it on a schedule with a unique version marker. Measure when that marker becomes retrievable through the real production query path.

This catches failures that job health checks miss: stale caches, delayed embedding queues, partial index commits, permission-filter drift, and replicas that have not received the update.

Test deletion too. Old information that remains retrievable after access revocation or document removal is a security issue, not merely a freshness issue. Measure time-to-removal with the same seriousness as time-to-add.

Run chaos exercises against the ingestion path. Pause a queue, expire a connector token, send a malformed document, and make one index replica lag. Verify that alerts identify the affected scope and that the answer path enters the intended degraded mode.

Design for Recovery, Not Just Alerts

When lag grows, blindly increasing workers can amplify rate limits and duplicate processing. Use idempotent jobs keyed by source and version. Separate detection from processing so a failed parser does not lose the fact that a change exists. Maintain dead-letter queues with a clear owner and age metric.

Backfills should prioritize risk and recency rather than processing an enormous queue in arrival order. A current security policy matters more than the 2019 revision of a low-traffic brochure.

Finally, keep generation caches version-aware. A perfectly fresh index is useless if the answer layer returns a cached response created from an old evidence set. Include collection revision or source-version dependencies in the cache key.

RAG is a data system wearing an AI hat. Semantic relevance gets the demos, but freshness determines whether people can trust the answer after the source changes. Put a clock on the pipeline, define the failure behavior, and test it from source mutation to final response.