RAG Authorization and Tenant Isolation Testing

Short answer

RAG access control is not achieved by telling a model to “use only permitted documents.” A trusted backend must enforce document permissions before retrieval, then preserve that context through cache, reranking, citations, history, and generation. Success means unauthorized content never reaches the model or leaks through adjacent channels.

RAG copies source documents into chunks, embeddings, and indexes. ACL updates can drift between the source and search copy. Tenant or group-membership errors can expose data through answers, citations, titles, caches, or conversation history.

Identify the authoritative permission source

For each document, establish whether the source application, synchronized group membership, or a policy service owns its ACL. User roles must not come from a prompt or client metadata. Resolve tenant context server-side from the authenticated request.

If ingestion copies tenant and permission tags into index metadata, document update guarantees. Retrieval must not search every tenant and ask the prompt to separate results; apply the permission filter before constructing model context.

Trace ACL lifecycle

How quickly does group removal, document permission change, or tenant suspension reach the index? If the source is current while the index retains old permissions, an authorization window exists. Test failed delta updates and old copies during reindexing.

Deletion means more than disappearance from search. Content may remain in conversation history, semantic or reranker cache, exports, or citation URLs. Verify that access is removed from every store and that completion can be observed.

Chunks, caches, and citations

Source ACLs can be correct while chunk metadata is lost. Verify each chunk retains tenant and permission labels. When several documents are combined, assess access per source rather than widening to the broadest permission.

Partition cache keys by tenant, user, or permission version. After Tenant A asks a query, Tenant B asking the same query must not receive A’s answer. Measure when an ACL change invalidates old cache entries.

Citations are a data-exposure surface. An answer might omit the document while still exposing a title, file path, customer name, or snippet. If a user cannot open the source, retrieval results and citations should not reveal it either.

Build a test matrix

Prepare synthetic, unique canary phrases for every tenant and group. Ask the same query as authorized and unauthorized users through direct retrieval, natural language, follow-ups, history, another language, and a warmed cache. Check that unauthorized canaries never enter model context, answers, citations, or user-visible log output.

In a test environment, exercise role changes, tenant switching, group removal, ACL updates, source outages, index rebuilds, cache clearing, and deletion. Define an acceptable consistency window for each transition as a product requirement.

Architecture checklist

• Authenticated backend context makes access decisions, not the model.
• Tenant and ACL filters run before retrieval.
• Each chunk retains source, tenant, permission version, and deletion state.
• ACL changes invalidate relevant caches.
• Policy-service failures never fall back to unfiltered retrieval.
• Citations and conversation history follow the same access policy.

Make synchronization delay measurable

“Permissions update shortly” is not a testable guarantee. Trace how a source ACL change triggers an event, ingestion queue, index update, and cache invalidation. Retries and failures at every step should be observable. If the product accepts a window during which a removed user retains access, define that duration explicitly and set tighter limits for highly sensitive documents.

During a rebuild, keeping the old index active until the new one is ready can be reasonable, but the old index must not preserve revoked permissions. Blue/green index swaps should match an ACL snapshot version and occur atomically. If rollback restores an old index or cache, verify that it does not restore old access rights too.

Turn test data into a leakage trace

Use unique synthetic canaries per tenant and group. For each query, correlate the chunk IDs sent to the model, tenant filters, cache hit or miss, and returned citation IDs in safe telemetry. This can reveal a chunk that reached model context even when the generated answer happened not to repeat its sensitive text.

A useful report records when a permission changed, how long retrieval returned stale content, the cache’s role, and which layer needs remediation. A screenshot of the final answer is insufficient: the model’s choice not to repeat sensitive material does not prove that access control worked.

A retest must prove more than a model refusing one prompt: unauthorized content should never enter retrieval results. Verify tenant crossover, ACL delay, cache behavior, and deletion separately. For a review of RAG retrieval and backend boundaries, see our AI security assessment.