85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is contextual retrieval?

Contextual retrieval is a technique, published by Anthropic in September 2024, that prepends a short model-written explanation of each chunk's place in its source document before the chunk is embedded and indexed for keyword search.

Checked

Retrieval systems split documents into chunks, and a chunk on its own often loses the facts that made it findable. A line that says the limit was raised to 50 units does not say which product, which customer tier or which year.

Contextual retrieval fixes this at indexing time. A model reads the whole document and each chunk, writes a short note that situates the chunk, and that note is stored with the chunk in both the vector index and the BM25 index.

How it works

  • Generate the context

    In Anthropic's setup, Claude 3 Haiku received the full document and one chunk, and wrote a context that is usually 50 to 100 tokens long.

  • Prepend and index

    The context is added in front of the chunk before embedding and before building the BM25 index, so both searches see it.

  • Keep cost down with caching

    The document is cached once and reused for every chunk. Anthropic estimated a one-time cost of about $1.02 per million document tokens with prompt caching.

What Anthropic measured

Across several domains, with Gemini Text 004 embeddings, the top-20 retrieval failure rate fell from 5.7% to 3.7% with contextual embeddings, to 2.9% when contextual BM25 was added, and to 1.9% when a reranker was added on top.

The same post notes a simpler option: a knowledge base under about 200,000 tokens, roughly 500 pages, can go straight into the prompt with prompt caching and skip retrieval altogether.

How Motiontic's engineers apply it

Motiontic's engineers try contextual retrieval when evals show chunks failing because they lack names, dates or section headings that only appear elsewhere in the document. The context prompt is tuned per document type, such as contracts, tickets or manuals, and re-indexing cost is estimated before the corpus is reprocessed.

Anthropic's Contextual Retrieval results (top-20 retrieval failure rate)

Anthropic's Contextual Retrieval results (top-20 retrieval failure rate)
ConfigurationFailure rateReduction
Baseline embeddings5.7%None
Contextual embeddings3.7%35%
Contextual embeddings and contextual BM252.9%49%
Both, plus a reranker1.9%67%

Common questions

What is the difference between contextual retrieval and standard RAG?

Standard RAG indexes chunks as they are. Contextual retrieval first adds a short, model-written description of where each chunk sits in its document, so the chunk can be found by queries that mention context it does not contain.

How much does contextual retrieval cost?

Anthropic estimated about $1.02 per million document tokens as a one-time indexing cost, using prompt caching so the full document is not paid for again for every chunk.

When is retrieval not needed at all?

Anthropic's post notes that a knowledge base under about 200,000 tokens can be placed directly in the prompt, with prompt caching to cut cost and latency.

Does contextual retrieval work with any embedding model?

The method changes the text that gets embedded, not the model, so it works with any embedding model and any BM25 index. Results vary by corpus and should be checked on an eval set.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.