RAG explained: answers grounded in company knowledge

A general language model has capabilities and knowledge acquired during training. It does not automatically know your current maintenance manual, individual contract version or internal access permissions. Fluent wording can conceal that missing access.

RAG adds information from a designated collection at runtime. The original research combined a trained model with an external information store. That work is an important technical foundation, but its experimental results are not a performance promise for a present-day company assistant. Lewis et al.: retrieval-augmented generation.

The application still needs decisions about which sources it may search, which version applies and who is asking. The label RAG does not answer those organizational and technical questions.

This scenario is fictional. A service team supports two similar machine models. Model A requires a different maintenance sequence from Model B. An employee asks which steps precede replacing the filter on Model A.

The application recognizes the model identifier, searches the approved documentation and finds the relevant section. It supplies that section to the language model together with its title, version and material safety instructions. The model produces an understandable answer with a source reference.

If search finds only the manual for Model B, the assistant must not transfer the procedure automatically. If a safety-critical prerequisite is missing, it should identify the gap or direct the employee to the complete instructions. The correct outcome may be a clarification question even when the software could technically generate an answer.

The following table lists the five RAG steps with what happens and what needs checking.

The first two stages often run independently of any question. Source changes must subsequently reach the search collection. The last three occur during use. An ingestion failure can therefore affect today's answers while the chat interface appears perfectly healthy.

Chunk. A chunk is a section of content the search system can retrieve independently. It needs enough context to remain understandable. Separating a paragraph from its heading or exception can change its meaning.

Embedding. An embedding is a numerical representation of content. Search systems can use it to compare meaning even when different words are used. That representation is neither proof of truth nor an automatic access-control mechanism.

Vector database. A vector database stores those representations and supports similarity search. It is one possible component rather than the whole RAG application. Conventional text search and metadata filters are often also useful, especially for exact product identifiers or effective periods.

Reranking. Reranking reorders candidates already retrieved. It can promote a relevant passage, but cannot use a source that was never found. The hybrid-search guide explains this selection process in more detail.

A chat interface can use a general model, ordinary search, RAG or a combination of tools. Its appearance does not reveal its architecture. A file-upload button also says little about long-term storage, permissions or update behavior.

RAG is not restricted to custom software. Microsoft describes retrieval and answer-generation workflows in Copilot Studio. As with custom applications, the important details are the actual connected sources and their properties. Microsoft: RAG in Copilot Studio.

When selecting a product, ask about its actual data flow. Where does an answer come from, whose identity is used for retrieval and how does a deleted source disappear? An architecture label is not a substitute for those answers.

Search and RAG. Ordinary search returns source results. RAG adds a synthesized answer. That can help when multiple passages must be combined or technical language explained. If employees simply need to open a known file, a good search interface may already solve the problem.

Fine-tuning. Fine-tuning changes a model through additional training. It can be relevant for recurring task behavior or output style. It does not automatically replace access to current documents under the caller's permissions. Both approaches can be combined, but first identify the failure you need to fix.

Direct database query. A direct database query is often more appropriate for exact quantities, totals or transaction status. Today's inventory may require a current system query. A passage extracted from an older PDF is not an equivalent source for that question.

An answer can be wrong even when a citation exists. The source may be obsolete, the passage may describe another product variant or the model may draw an unsupported conclusion. Check material claims against the actual passage used.

Privacy and access control also do not arise automatically from RAG. Restrict documents before they reach answer generation. Sources may contain malicious instructions that must not acquire the authority of system commands.

Knowledge needs maintenance as well. When responsibilities or policies change, the application must ingest new versions and handle older ones according to their validity. An assistant without accountable source ownership will deteriorate even if its model remains capable.

Recurring questions, reliable existing documentation and meaningful search or explanation effort provide a useful starting point. A domain where knowledge exists only in people's heads or every decision requires substantial individual judgment is less straightforward.

Collect real questions and their authoritative sources first. Check whether existing search or an already deployed product meets the need. A bounded RAG trial can then address the remaining gaps.

Include correct answers, missing knowledge, wrong versions and different reading permissions in that trial. Measure time to a verified solution, including checking and rework. The RAG acceptance plan provides a starting structure; the architecture guide covers technical choices once the need is established.

A useful first outcome may be a better source collection rather than a new assistant. Discovering that several supposedly authoritative documents disagree is valuable evidence. Resolve that disagreement before expecting a language model to turn it into dependable operational guidance.

These modernagents guides cover related decisions about AI knowledge assistants:

• AI knowledge assistants: sources, access and answers
• RAG architecture from source document to checked answer
• RAG or fine-tuning: choose by the problem
• Build an AI knowledge base with owners and current sources

For a workflow review, start with the task, systems involved and typical exceptions. Discuss it with modernagents.