What is RAG?

RAG: definition and core idea

A large language model only knows what was in its training data. Internal price lists, current product data or your work instructions are not part of it. If you ask about them anyway, in the worst case the model makes up a plausible-sounding answer.

RAG solves this problem by combining two steps: first search (retrieval), then formulate (generation). The method was described in a research paper in 2020 and is now a widespread way of connecting AI applications with company knowledge. The language model itself remains unchanged—the knowledge is supplied fresh with every request.

How retrieval-augmented generation works

First, the documents are prepared: split into meaningful sections and stored as so-called embeddings—numerical vectors that represent the meaning of a text. These are stored in a vector database, which can be updated automatically when documents are added or changed. Each question then triggers a fixed sequence of a few steps.

• The user’s question is also converted into a vector
• The search finds the text sections that best match in content, often combined with classic keyword search
• The results are passed to the language model together with the question
• The model formulates an answer based on these sources and can cite them

Benefits and limits of RAG

RAG makes AI answers more up to date, more verifiable and specific to your company. New documents are available immediately without retraining the model. Source citations allow users to verify statements. Access rights can also be taken into account, so that everyone only receives answers from documents they are allowed to see.

However, quality stands or falls with the data and the search. Outdated or contradictory documents lead to wrong answers, and if the search doesn’t find the right section, even the best model can’t help. Errors can’t be ruled out entirely—so testing with typical questions is part of every project.

Real-world example: RAG in technical service

A machine manufacturer has extensive operating manuals and service notes. A RAG-powered assistant answers questions from service technicians such as “What tightening torque applies to this component?” directly from the manuals and links to the relevant passage. Beforehand, the documents are cleaned up and outdated versions are sorted out so that the assistant doesn’t quote from obsolete documents.

Instead of leafing through pages for a long time, the technician quickly receives an answer along with its source. Nano Concepts designs and develops RAG applications—from data preparation and vector search to integration into your website, intranet or app. New manual versions are incorporated automatically.