Enterprise RAG: retrieval, relationships and trustworthy evidence

An employee asks, “Does policy TR-17 cover this trip?” The company has a public travel handbook, a manager-only compensation annex and an amendment that took effect last month. Finding text that resembles the question is easy. Returning the applicable evidence without exposing the annex is the real engineering problem. Enterprise RAG is an evidence pipeline with access and version controls, not merely a vector search followed by a prompt.

Build an evidence pipeline#

The 2020 RAG paper distinguishes a parametric model from an external knowledge index. In a business application, that index may combine exact IDs, SQL, lexical search, vectors and relationships. Retrieval selects evidence from external sources; it does not retrieve hidden facts from inside the LLM. A generator then explains what the selected evidence supports.

Authorize before evidence reaches the model#

Determine allowed organizations, records and source versions using the authenticated caller. Apply those constraints before candidates, snippets or derived summaries enter generation, shared logs or response caches. Filtering the final answer cannot undo private text already sent to an external model. Source access should also be checked when a citation is opened, because permission may have changed since answer generation.

Database row-level security is one enforcement option, with important role behavior. PostgreSQL documents that table owners normally bypass policies and superusers or BYPASSRLS roles bypass them. Test the actual application role, workers and maintenance paths. A query written with a tenant filter is useful, but a privileged background process can still leak data if it omits that filter.

Azure’s query-time ACL feature illustrates a service-managed approach, but the checked documentation labels it preview and describes propagation and source limitations. Do not treat an indexed permission snapshot as guaranteed current authorization. Define a maximum permitted lag, use an authoritative check for sensitive reads, and block access immediately when a revocation cannot safely wait for reindexing.

A citation must support its claim#

A plausible URL or a topically related paragraph is insufficient. Link a claim to the passage that actually supports it, including the exception or effective date that makes it applicable. The ALCE citation research treats citation quality separately from fluent generation. For this design, evaluate support at the claim level and make source text inspectable.

If the handbook describes domestic travel but the question concerns a cross-border assignment, the correct answer may be that the available evidence is insufficient. State the missing condition and the next authorized review path. Do not reveal that a restricted document exists as a way of explaining the refusal. Distinguish a source’s explicit statement from your system’s inference about the user’s situation.

Track every derivative of a source#

A document produces chunks, embeddings, summaries, graph relations and cached answers. Removing the original file does not necessarily remove those derivatives. Azure’s deletion-handling documentation explicitly describes orphan index documents when deletion detection is absent. Keep a source-to-derivative manifest and a tombstone state that stops serving affected material while cleanup proceeds.

A replacement should create a new source version with an effective interval. Rebuild or invalidate the derivatives that depend on the old version, and record completion. Audit whether a historical question may still use a retained old version. Deletion, retention and legal hold are separate policies; engineers should implement the organization’s approved rules rather than silently retaining everything for convenience.

Start with exact and hybrid retrieval#

Exact policy codes, supplier names and clause numbers often need lexical or structured matching. Semantic retrieval helps with paraphrases. Hybrid search combines candidates from both. Azure documents reciprocal-rank fusion as one way to combine ranked lists: it uses rank position rather than treating unlike raw scores as directly comparable. The fused score is not a probability that the answer is true.

Evaluate chunk boundaries before adding complexity. A chunk saying “this exception applies” loses meaning when detached from its entity or date. Anthropic’s contextual retrieval approach adds explanatory context before indexing. If you generate that context, preserve the original passage and mark the augmentation as generated; otherwise an incorrect summary can become invisible evidence contamination. Reranking may improve candidate selection but cannot recover a source never retrieved.

Add relationships for relationship questions#

An amendment chain is naturally relational: agreement → amendment → effective date → superseded clause. A supplier dependency question may also require structured links. Start with reliable database relationships where available. A graph extracted from prose adds a different uncertainty: the relationship itself may be wrong. Store the supporting passage and extraction version for every generated edge.

The GraphRAG paper studies corpus-level summarization through entity graphs and community summaries. Microsoft’s query documentation distinguishes local, global and DRIFT approaches. These are useful options for different questions, not proof that a graph improves every lookup. Global summaries can be expensive and can mix access scopes. A community report derived from private and public documents is itself sensitive; filtering only the final leaf citations is too late.

Use iterative retrieval only when findings change the next query#

A first search may discover that the travel exception is governed by a later regional policy. That observation can justify a second query. Bound the number of hops, allowed sources, time and cost; retain why each hop was requested. Stop when the required evidence is found, the next step repeats a failed query or the missing material is inaccessible. An agent must not interpret insufficient evidence as permission to search another tenant.

Compare designs on the same authorized questions#

Use a fixed authorized corpus and held-out questions. Measure answer-bearing retrieval recall, ranking precision, supported-answer quality, citation coverage, wrong-version answers and unauthorized exposure. Include exact-code, paraphrase, multi-hop, global-summary and absent-answer cases. Report latency and cost per successful task alongside quality. A lower average latency is not a win if it drops the exception clauses that determine the answer.

Separate capture time from applicability#

A file ingested today may describe a rule effective last year. Keep source ID, version, content hash, effective interval where known, ingestion time and source authority separately. If the effective date is unknown, preserve that uncertainty. Do not infer that the newest upload automatically supersedes an older signed agreement.

Keep the path back to the original#

Each evidence item should point to a document version and a page, section or record locator. Record transformations such as OCR, normalization, chunking, generated context and summarization with their versions. A content hash detects changes to captured bytes; it does not prove that the source is authentic or that its statement is correct. Retain snapshots only where the access and retention policy permits them.

Represent disagreement directly#

If two sources disagree, present their respective dates, authority and supporting passages. A deterministic rule may establish precedence; otherwise leave the conflict unresolved for review. Missing evidence is a separate state from a negative finding. “No permitted document found” cannot safely become “no obligation exists.” Capture this distinction in the answer schema and evaluation labels.

Deliver an evidence packet that survives review#

An audit-ready packet for the illustrative travel question contains the question and scope, applicable policy version, source locators, supported claims, unresolved exceptions, transformation lineage and reviewer decision. Keep the packet access-controlled; a saved answer must not become a permanent public copy of a once-authorized source. Revalidation and retention apply to the packet as well as the search index.