RAG (Retrieval-Augmented Generation)

What does RAG mean in AI?

Break the name into its parts. Retrieval: find the right information. Augmented: add it to what the model sees. Generation: the model writes the answer. A large language model on its own only knows what it learned in training, up to its knowledge cutoff. RAG gives it an open book to read from at the moment it answers.

The term comes from a 2020 research paper by Patrick Lewis and colleagues, which paired a language model with a searchable index of Wikipedia and reported more specific and more factual answers than the model alone.

Outside AI, RAG has other meanings: a rag is a cloth, and in project reporting RAG often stands for red, amber, green status. In AI, it means retrieval-augmented generation.

How RAG works

The analogy: a closed-book exam versus an open-book one. Without RAG, the model answers from memory. With RAG, someone hands it the right pages first.

The precise version, in the most common setup:

• Your documents are split into chunks, and each chunk is turned into an embedding, a list of numbers that captures its meaning.
• The embeddings are stored in a vector database.
• When a question arrives, it is embedded too, and a semantic search returns the closest chunks.
• Those chunks go into the model's context window with the question, and the model writes an answer grounded in them.

A real example of RAG

A customer asks a support bot, "How long do I have to return a jacket?" Instead of guessing, the system retrieves the returns section of the help center, passes it to the model, and the bot answers from that text, ideally with a link to the source. When the policy changes, you update the document, and the next answer reflects it without retraining anything.

Why RAG matters and when to use it

RAG is the standard way to make AI answer from private or fast-changing information: help docs, policies, product sheets, manuals. Anthropic's documentation notes it is particularly useful for up-to-date information, domain-specific knowledge, and citing sources. It also reduces hallucination, because the model has real text to work from, though it does not eliminate it.

Use RAG when the problem is what the model knows. Use fine-tuning when the problem is how it behaves, such as a strict format or style.

Related terms

• Grounding: tying answers to checkable sources, which is the goal RAG serves.
• Embedding and vector database: the usual machinery behind the retrieval step.
• Hallucination: the confident wrong answers RAG is meant to reduce.
• Context window: the space the retrieved passages have to fit into.

Common misconceptions about RAG

• RAG trains the model on your data. It does not change the model at all; it changes what the model reads at answer time.
• RAG makes answers always correct. If retrieval pulls the wrong passage, or the documents are outdated, the answer inherits the problem.
• RAG requires a vector database. Embeddings are the most common method, but keyword search, a database query, or a web search can also be the retrieval step.
• A RAG agent is a different technology. "RAG agent" usually means an AI agent that uses retrieval as one of its tools.

RAG in practice

This page owns the short definition. For the full walkthrough, including RAG vs fine-tuning, real business uses, privacy, and how to start, read our deep guide: what is RAG (retrieval-augmented generation). The retrieval step usually runs on embeddings, so that entry is the natural next stop.