The decision this supports
Treat the knowledge source, the requester’s permissions, and the answer’s evidence as parts of one operating system.
For knowledge owners, architects, and service leaders
01 / Architecture guide
Start with the questions people need to answer
A knowledge assistant needs a narrower purpose than “chat with our documents.” Choose a role, a set of decisions, and the sources that role is permitted to use. For a service specialist, the task may be to identify which onboarding requirements apply to a client and explain what is still missing.
Collect representative questions, including ambiguous wording, follow-ups, and questions the system should decline. Ask a domain owner to identify the expected sources and the limits of an acceptable answer. This establishes the basis for evaluating retrieval before writing an elaborate prompt.
Retrieval-augmented generation uses retrieved material to inform a model’s response. Microsoft’s RAG architecture guidance identifies access control, source integration, relevance, and response time as design concerns. The architecture should address those concerns explicitly; adding document search does not by itself establish a trustworthy answer.
02 / Architecture guide
Give every source an owner and a lifecycle
Identify who can say a source is authoritative, which version applies, how changes reach the assistant, and how withdrawn material is removed. Start with a bounded, maintained collection. Indexing more material can introduce duplicate policies, obsolete procedures, and unresolved conflicts alongside useful context.
Scroll across the table to read all columns.
| Record | Question it answers | Example |
|---|---|---|
| Owner and purpose | Who can confirm this source belongs here? | Service operations owns the onboarding procedure. |
| Scope and effective date | When and to whom does it apply? | A regional procedure effective from a stated date. |
| Access rule | Which requesters may retrieve it? | Only members of the relevant service team. |
| Update and removal path | How do changes reach every copy? | Refresh the index and invalidate affected cached material. |
| Conflict rule | What happens when sources disagree? | Show the conflict and ask the policy owner to resolve it. |
Store enough provenance to connect a retrieved passage to its source and version. Decide how tables, attachments, and document boundaries will be represented. A useful paragraph can lose its meaning if the qualifying heading or effective-date note is separated from it.
03 / Architecture guide
Apply permissions before generating an answer
Carry the requester’s identity through retrieval and enforce authorization in trusted application or data services. A prompt telling the model to ignore restricted material is not an access-control boundary. Review what appears in retrieval results, citations, caches, and logs as well as the final response.
Test two users asking the same question with different permissions. Include revoked access, a moved document, and a deleted source. Decide how quickly access changes must take effect and whether the chosen architecture can meet that requirement across all retained copies.
Keep retrieved text separate from trusted system instructions. A document can contain misleading statements or attempts to direct an assistant’s behavior. Limit tools to the task, validate their arguments, and require the appropriate approval before an answer becomes a consequential action.
04 / Architecture guide
Inspect retrieval separately from answer quality
When an answer is wrong, first ask whether the correct evidence was available and retrieved. If it was absent, changing the response prompt may hide the symptom without correcting the source or search problem. If it was present, examine whether the answer followed the evidence and preserved qualifications.
Measure the system on complete, missing, outdated, conflicting, and inaccessible evidence. Check that citations support the claim beside them; a valid link to an unrelated passage is not sufficient. Test questions spanning multiple sources and questions for which the correct response is that the available material cannot establish an answer.
Use domain review to judge usefulness. A technically grounded answer can still omit the distinction that matters to the next decision. Ask whether the reader can understand the applicable rule, inspect its source, and see what requires a person’s judgment.
05 / Architecture guide
Operate the collection as a business capability
Assign ownership for reported gaps, source changes, access incidents, and model or retrieval updates. Make feedback actionable: retain the question, permitted evidence, answer, and reviewer’s correction within the agreed information policy. Route a source defect to its owner instead of treating every complaint as a model problem.
Track time to find and validate information, unresolved questions, source freshness, and correction effort. Review demand for unsupported topics before widening the collection. Expansion should bring a new source owner, evaluation cases, and an explicit access decision with it.
A sensible first release may only prepare an answer with citations for a person to inspect. Add system actions only when the team can define the authority, acceptance criteria, and recovery path for those actions. Useful knowledge access does not require immediate autonomous execution.