What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation (RAG) is a method in which a system fetches relevant passages from a company's own documents or data at query time and places them in the model's prompt, so answers draw on current, citable sources.
Checked
A language model only knows what was in its training data and what is in its prompt. RAG fills the prompt with the passages that matter for each request, pulled from the organisation's own documents, tickets or records, so the model answers from current facts and can point to where they came from.
The term comes from a 2020 paper by Patrick Lewis and co-authors, which combined a pre-trained generator, the parametric memory, with a dense vector index queried by a neural retriever, the non-parametric memory.
The pipeline
Ingest and chunk
Documents are parsed, cleaned and split into passages, with titles, dates and permissions kept as metadata.
Embed and index
Passages are embedded into a vector index and usually indexed for keyword search too.
Retrieve
Each query runs hybrid search, filtered by what the user is allowed to see.
Rerank
A reranker reorders the shortlist so the most relevant passages reach the model.
Generate with citations
The model answers from the passages and cites them. Anthropic's search_result content blocks let Claude cite retrieved chunks the way it cites web results.
Build or use a managed service
The labs now sell managed retrieval. OpenAI offers a file_search tool in the Responses API over managed vector stores, Google launched File Search in the Gemini API on 2025-11-06 with chunking, embeddings and citations handled, and Vertex AI RAG Engine offers a managed store or a bring-your-own one. These cut setup time for small and medium document sets, at the cost of control over chunking and ranking.
A custom pipeline earns its place when permissions are complex, documents need special parsing, or evals show the managed defaults miss too often.
RAG inside agents
In an agent, retrieval becomes a tool the agent can call several times rather than a single step before the answer. The RAG vs agentic search page covers when an index helps and when file tools are enough, and the RAG vs fine-tuning page covers when to retrain instead.
How Motiontic's engineers apply it
Motiontic's engineers build the eval set before the pipeline: real questions, the passage that answers each one, and the answer expected. Retrieval and generation are scored separately, so a wrong answer can be traced to a missed passage, a bad ranking or a model that ignored good context.
Where RAG pipelines fail, and the usual fix
| Stage | Common failure | Usual fix |
|---|---|---|
| Chunking | Passages lose the names and dates that made them findable | Contextual retrieval, better chunk boundaries |
| Retrieval | Exact codes or paraphrased questions missed | Hybrid search |
| Ranking | The right passage is retrieved but ranked too low | A reranker |
| Permissions | Users see passages they should not | Filters applied at retrieval time |
| Generation | Answer ignores or contradicts the passages | Citation requirements and graded evals |
Common questions
What does RAG stand for?
Retrieval-augmented generation. The system retrieves relevant passages first, then the model generates an answer using them.
When is RAG not needed?
When the knowledge fits in the prompt. Anthropic's Contextual Retrieval post notes that a knowledge base under about 200,000 tokens can go straight into the prompt, with prompt caching to cut cost.
Is RAG the same as fine-tuning?
No. RAG changes what the model sees for each request. Fine-tuning changes the model's weights. RAG suits knowledge that changes; fine-tuning suits behaviour that should stay fixed.
Is RAG still relevant for AI agents?
Yes, as a tool. Agents call retrieval when they need it, sometimes several times per task, and indexed hybrid search remains the stronger option for large or conceptual document sets.
Related
RAG vs Fine-Tuning
Fetching the right facts at query time, compared with retraining the model's behaviour.
RAG vs Agentic Search
One retrieval before the answer, compared with an agent that searches as it works.
Hybrid search
Keyword ranking and vector similarity, fused into one result list.
Reranker
A second pass that rereads the shortlist and puts the right passages first.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Query rewriting
Fixing the question before searching, so retrieval has something to match.
Sources
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, 2020)
- Anthropic, Introducing Contextual Retrieval (2024-09-19)
- Claude Platform docs, Search results
- OpenAI, File search tool guide
- Google, File Search tool in the Gemini API (2025-11-06)
- Google Cloud, Vertex AI RAG Engine vector database choices
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.