Permission-Scoped Retrieval Indexes: Keeping Multi-Tenant RAG From Leaking Across Boundaries

Here’s a scenario that plays out in more companies than anyone likes to admit: your RAG assistant works beautifully in the demo. Ask it about a contract, and it cites the right clause. Then a customer support lead asks a question and the answer quotes a document from a completely different account. Nobody notices for three weeks. When someone finally does, you spend a weekend auditing logs and realize the problem was never the model — it was the retrieval layer.

If you’re building multi-tenant RAG, retrieval permission scoped indexes are the difference between a system you can ship to enterprise customers and one that quietly fails its first security review. This tutorial walks through why naive retrieval leaks, the architectural patterns that prevent it, and a working implementation you can adapt to Postgres/pgvector, Weaviate, Pinecone, or Qdrant.

We’ll cover the design decisions that matter, write code for the enforcement layer, and walk through the pitfalls that bite teams long after launch.

---

Prerequisites

Before you start, you should have:

• Working knowledge of RAG pipelines — chunking, embedding, retrieval, and prompt construction. If you’ve built even a toy RAG app, you’re fine.
• A vector store you can run locally or in a sandbox — this tutorial uses Postgres with pgvector because it keeps the permission data next to the vectors, but the patterns translate to dedicated vector databases. Weaviate’s multi-tenancy docs and Pinecone’s metadata filtering are worth reading alongside this.
• Familiarity with SQL and a web framework — examples use Python and FastAPI-style pseudocode, but nothing here is framework-specific.
• An identity model — you need to know who the caller is and what tenants/roles they belong to. This tutorial assumes you already issue authenticated tokens with tenant claims.

You do not need a fancy embedding model. Any standard embedding pipeline works; the security properties come from the index design, not the model.

---

Why shared indexes leak in the first place

Most RAG tutorials show a single collection with all documents embedded together. At query time, you embed the user’s question, run a similarity search, and stuff the top-k chunks into the prompt. That’s fine for a personal notes app. It’s a liability the moment two organizations share the collection.

There are three ways leaks happen, and they’re worth naming explicitly because each requires a different defense.

Leak vector one: the missing filter

The most common failure is simply forgetting to filter. A developer writes query(embedding, top_k=5) and ships it. Every user’s question searches every tenant’s documents. The model dutifully quotes a competitor’s pricing sheet, and nobody blames the retrieval layer because the answer sounds plausible.

Leak vector two: filter-after-retrieval

A subtler bug: you retrieve the global top-k, then filter results by tenant in application code. This looks safe — the final prompt only contains permitted chunks — but it has two problems. First, you may end up with fewer than k results (or zero) because the top matches were all from other tenants, which degrades answer quality unpredictably. Second, if you log or cache the raw retrieval results anywhere, you’ve already exposed cross-tenant data to your own infrastructure. It also wastes embedding compute and latency on documents the user can never see.

Leak vector three: metadata drift

Even teams that filter correctly get burned when permission metadata goes stale. A user is removed from a project, but their cached session still carries the old role. A document is re-chunked and the new chunks are missing the tenant_id field. A backfill job writes vectors without the permission columns because it used the wrong code path. The filter exists, but it’s filtering on the wrong data.

The fix for all three is the same in spirit: permissions must be structural properties of the index and enforced inside the query, not applied as an afterthought.

---

The three architectures, ranked

There are three broad ways to scope retrieval, and they differ sharply in isolation guarantees and operational cost. Choose deliberately.

1. Index-per-tenant (strongest isolation, highest overhead)

Each tenant gets its own collection, namespace, or even database. Queries never touch another tenant’s data because the query physically can’t reach it.

Weaviate’s multi-tenancy model is built around this: tenants are isolated units within a class, and a query against one tenant’s data cannot return another’s. Pinecone namespaces and Qdrant collections offer similar isolation.

When to use it: regulated industries, enterprise contracts that require demonstrable data separation, or tenants large enough to justify dedicated resources. It’s also the easiest architecture to reason about during a security audit.

The cost: thousands of small indexes are expensive to operate. Cross-tenant admin queries (e.g., “search all tenants for this phrase”) become awkward. Schema migrations and re-embeddings must run per tenant.

2. Shared index with mandatory permission filters (best balance for most SaaS)

One index, but every vector carries tenant and role metadata, and every query must include a filter on that metadata. This is the pattern most mid-size SaaS products should adopt: it’s cost-efficient, supports cross-tenant admin tooling, and — done correctly — is safe.

The catch is that “done correctly” is doing a lot of work. You need to make it impossible to query without a filter, keep metadata fresh, and test the enforcement continuously. Most of this tutorial focuses on making that safe.

3. Hybrid: partitioned by tenant tier

Large tenants get dedicated indexes; small tenants share a partitioned index with strict filters. This is what many mature products converge on. It’s also the most complex to operate, so don’t start here unless you have evidence you need it.

For the rest of this tutorial, we’ll implement architecture #2 properly, with notes on where #1 and #3 change the design.

---

Designing the permission schema

Before writing a line of retrieval code, define what a permission is in your system. A vector’s access metadata should answer: who can see this chunk, and under what conditions?

A workable schema looks like this:

CREATE TABLE documents (
  id UUID PRIMARY KEY,
  tenant_id UUID NOT NULL,
  source_uri TEXT NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE chunks (
  id UUID PRIMARY KEY,
  document_id UUID NOT NULL REFERENCES documents(id),
  tenant_id UUID NOT NULL,
  content TEXT NOT NULL,
  embedding vector(1536),
  -- permission scope
  visibility TEXT NOT NULL CHECK (visibility IN ('tenant', 'group', 'private')),
  allowed_groups UUID[] NOT NULL DEFAULT '{}',
  allowed_users UUID[] NOT NULL DEFAULT '{}'
);

CREATE INDEX ON chunks (tenant_id);
CREATE INDEX ON chunks USING ivfflat (embedding vector_cosine_ops);

A few design decisions are baked in here, and each one matters:

• tenant_id is denormalized onto chunks. Yes, it duplicates data. It also means the filter is always available at query time without a join, which is exactly what you want for a security-critical field. Joins in the hot path are where filters get dropped.
• Visibility is an enum, not a boolean. Real permission models have at least three tiers: whole-tenant, group-restricted, and private. Modeling this explicitly prevents the “everyone can see everything” default that sneaks in when the schema only supports public/private.
• Groups and users are arrays. This makes filtering possible in a single query. In Postgres, array containment operators (&&, @>) let you ask “does this chunk’s allowed set intersect the caller’s set?”

If you’re on a vector database without relational features, you’ll store the same fields as metadata and use the database’s filter syntax. Weaviate supports filtering on array properties; Pinecone supports metadata filters on string and list fields. The schema translates; the discipline doesn’t change.

The rule that makes this safe

Write this down somewhere your team will see it: no retrieval call may execute without a tenant filter. Not in admin tools, not in evaluation scripts, not in the “quick debug” endpoint someone added on a Friday. If you enforce one rule, enforce this one.

---

Building the enforcement layer

Application-level discipline fails eventually. The right move is to make the unsafe path impossible to express. We’ll build a retrieval service that takes a signed identity context and constructs the filter itself — callers never pass raw filters.

Step 1: Define the caller context

from dataclasses import dataclass
from uuid import UUID

@dataclass(frozen=True)
class CallerContext:
    user_id: UUID
    tenant_id: UUID
    groups: frozenset[UUID]

This object should be produced only by your authentication middleware, derived from verified token claims — never from request body parameters. If a client can pass tenant_id in the query string, a client can lie about it.

Step 2: Build the filter from the context

def build_permission_filter(ctx: CallerContext) -> str:
    # Tenant boundary is unconditional.
    # Visibility tiers are OR-ed: a chunk is visible if it's tenant-wide,
    # or if the caller's groups/users intersect the chunk's allow lists.
    return f"""
      tenant_id = '{ctx.tenant_id}'
      AND (
        visibility = 'tenant'
        OR (visibility = 'group' AND allowed_groups && ARRAY[{group_list}]::uuid[])
        OR (visibility = 'private' AND allowed_users && ARRAY['{ctx.user_id}']::uuid[])
      )
    """

In production, use parameterized queries rather than f-strings — the interpolation above is for readability. The important structural point is that tenant_id is a hard AND, not part of the OR chain. A bug in the visibility logic can only over-restrict within the tenant; it can never cross the tenant boundary.

Step 3: The retrieval function

async def retrieve(ctx: CallerContext, query_embedding: list[float], k: int = 8):
    filter_clause = build_permission_filter(ctx)
    rows = await db.fetch(f"""
      SELECT id, content, 1 - (embedding <=> $1) AS score
      FROM chunks
      WHERE {filter_clause}
      ORDER BY embedding <=> $1
      LIMIT $2
    """, query_embedding, k)
    return rows

Note what’s absent: there is no path to call retrieve without a CallerContext. That’s the whole point. If your framework allows it, make the context a required positional argument and add a test that fails if anyone adds a default.

Step 4: Enforce at the database layer too

Application code is one defense. Row-level security is another. In Postgres, you can add an RLS policy as a backstop:

ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON chunks
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

Then set app.tenant_id on each connection from the authenticated context. Now even a raw SELECT * FROM chunks from a careless script is scoped. This is defense in depth: the application filter handles visibility tiers, and RLS guarantees the tenant boundary even if the application layer has a bug.

If you’re on a vector database without RLS, the equivalent is a strict rule that all access goes through your retrieval service, plus integration tests that assert cross-tenant queries return nothing.

Step 5: Expected result

At this point, a query from tenant A returns only tenant A’s chunks, further filtered by visibility. A query from a user in no groups returns only tenant-wide and private-to-them chunks. A query from a user in the “legal” group returns tenant-wide, legal-group, and their private chunks.

Test all three cases before moving on. If any case returns a chunk from another tenant, stop and fix the schema — don’t proceed.

---

Keeping permission metadata fresh

The retrieval filter is only as good as the metadata it reads. Stale permissions are the leak vector that survives correct filtering code, so treat metadata freshness as a first-class engineering problem.

Invalidate on permission change

When a user is removed from a group, or a document’s visibility changes, the change must propagate to the index. Two approaches:

• Re-write affected vectors. For a group membership change, find all chunks whose allowed_groups includes that group and update them. This is precise but can touch many rows.
• Resolve permissions at query time. Instead of storing allowed_groups on chunks, store group IDs as tags and resolve the caller’s group membership from your identity provider on each request. This is fresher by construction but adds a lookup to the hot path.

Most teams end up doing a mix: store the allow lists on chunks for fast filtering, and run a reconciliation job that re-syncs from the source of truth on a schedule (hourly is typical; minutes for high-sensitivity tenants). Log every reconciliation run and alert if the lag exceeds your SLA.

Never trust cached sessions for permissions

If you cache user sessions, cache the identity (user ID, tenant ID) but re-resolve group membership per request or with a short TTL. A session that outlives a permission revocation is a leak waiting to happen. The same applies to cached retrieval results: key your cache on (tenant_id, user_id, groups_hash, query) so a permission change invalidates the entry.

Backfills are where filters die

The most common production incident in this space is a backfill or migration script that writes chunks through a code path that doesn’t set permission fields. Enforce this at the schema level: make tenant_id, visibility, and the allow lists NOT NULL with no defaults, so a missing value fails loudly at insert time rather than silently defaulting to “visible to everyone.”

---

Testing for leaks (this is not optional)

Security properties that aren’t tested decay. Build a leak test suite and run it in CI on every change to the retrieval layer.

A minimal suite:

1. Cross-tenant isolation. Seed two tenants with distinctive content (“alpha-only” and “beta-only”). Query as tenant A. Assert no result contains “beta-only.” Repeat in reverse.
2. Visibility tier enforcement. As a user outside the “legal” group, query for content that only exists in legal-group chunks. Assert zero results. Then add the user to the group and assert results appear.
3. Filter-after-retrieval regression. Add a test that asserts your retrieval returns exactly k results when the tenant has at least k permitted chunks, catching the bug where someone “optimizes” by retrieving globally and filtering later.
4. Metadata completeness. A test that inserts a chunk through every write path and asserts the permission fields are populated.
5. Backfill safety. Run your backfill on a test dataset and assert no chunk ends up with a null or default tenant.

If you use OWASP’s LLM Top 10 as a checklist, leak testing maps to the “sensitive information disclosure” category — worth citing in your security documentation when enterprise customers ask.

---

Common pitfalls and how to avoid them

Pitfall: using the wrong similarity metric for filtered search. When you filter heavily, approximate indexes can return fewer than k results because the nearest neighbors were filtered out. Postgres’s ivfflat and HNSW indexes with filters can behave this way. Mitigate by increasing probes/ef_search when filters are selective, or by using exact search for high-sensitivity tenants where recall matters more than latency.

Pitfall: trusting client-supplied tenant IDs. Covered above, but it bears repeating. The tenant must come from a verified token, never from the request payload.

Pitfall: prompt injection that requests other tenants’ data. Even with correct retrieval, a model can be tricked into believing it has access to data it doesn’t. The fix is simple: the model only ever sees what retrieval returns. Never give the model a tool that queries the database directly with model-chosen filters — or if you do, validate the filter against the caller’s context before executing.

Pitfall: logs and traces. Your retrieval logs, cache keys, and observability traces should be tenant-scoped too. A shared trace that includes chunk contents from another tenant is a leak in the eyes of an auditor, even if the user never sees it.

Pitfall: evaluation datasets. When you evaluate retrieval quality, run evaluations per tenant or ensure the eval harness uses a CallerContext. A global eval that queries across tenants will report misleading recall numbers and mask real problems.

---

When to graduate to index-per-tenant

The shared-index-with-filters design is safe when implemented correctly, but it has a ceiling. Consider moving specific tenants to dedicated indexes when:

• A contract requires demonstrable physical separation of data.
• A tenant’s volume makes filtered queries slow for everyone else.
• A tenant needs a different embedding model or chunking strategy than the rest.
• Your security team wants isolation guarantees that don’t depend on application code.

You can run both architectures simultaneously: route each request to the right index based on the tenant’s tier. The retrieval interface stays the same; only the backend changes. Design your retrieval service with that seam in mind from day one.

---

The takeaway

Multi-tenant RAG leaks are rarely caused by a sophisticated attack. They’re caused by a missing filter, a filter applied too late, or stale permission metadata — three boring failures that a permission-scoped index design makes structurally hard to commit. Build the tenant boundary into the schema, enforce it inside the query, back it with row-level security where your database supports it, and test for leaks in CI.

If you take one action from this article, make it this: audit your current retrieval code for the string tenant_id. If it doesn’t appear in every query path, you have a leak — you just haven’t found it yet. Fix it before your customers do.