What is a reranker?
A reranker is a second-stage model that rescores a shortlist of passages returned by a first search, reading the query and each passage together, so the most relevant passages reach the top before the language model reads them.
Checked
First-stage retrieval has to be fast over millions of passages, so it compares precomputed vectors or keyword statistics. That speed costs precision. A reranker trades the other way: it is slower per passage, so it only looks at the shortlist, but it reads the query and the passage together and judges relevance far more finely.
For agents the payoff is fewer and better tokens. A reranked top five can replace a noisy top fifty, which leaves more of the context window for the task itself.
How a reranker works
Bi-encoder first
The first stage embeds the query and every passage separately, so passage vectors can be computed once and searched quickly.
Cross-encoder second
The reranker takes each query and passage pair as one input and outputs a relevance score. Nothing can be precomputed, which is why it runs only on the shortlist.
Listwise variants
Some rerankers read several candidates at once and rank them against each other rather than scoring each pair alone.
What the measurements show
Anthropic's Contextual Retrieval study used a Cohere reranker on top of contextual embeddings and contextual BM25. Together the three steps cut top-20 retrieval failures by 67%, from 5.7% to 1.9%, against 49% without the reranker.
Rerankers in use in 2026
Cohere
Rerank 4 launched on 2025-12-11. Cohere's site now announces Rerank 5 with a feature it calls Context Focus, pitched as sharper search with fewer tokens.
Voyage AI (MongoDB)
rerank-2.5 and rerank-2.5-lite, released in August 2025, follow natural-language instructions about what counts as relevant.
Jina AI (Elastic)
jina-reranker-v3 is a 0.6B listwise model released under CC-BY-NC-4.0, and jina-reranker-v3.5 followed on 2026-07-27, also with non-commercial weights. Licence terms matter before self-hosting.
How Motiontic's engineers apply it
Motiontic's engineers add a reranker when the eval set shows the right passage is being retrieved but not ranked high enough to reach the model. The shortlist size, the final cut and the latency budget are tuned together, because each extra candidate adds reranking time to every query.
First-stage retrieval compared with a reranker
| First-stage retrieval | Reranker | |
|---|---|---|
| Input | The query against the whole index | The query with each shortlisted passage |
| Precomputation | Passage vectors or keyword index built ahead | None, scored at query time |
| Speed per passage | Very fast | Much slower |
| Precision | Good recall, rough ordering | Fine-grained ordering |
| Typical role | Find candidates | Choose what the model reads |
Common questions
What is the difference between a reranker and an embedding model?
An embedding model turns the query and each passage into separate vectors that are compared later. A reranker reads the query and a passage together and scores how well they match, which is more accurate but too slow to run over a whole index.
Is a reranker worth the extra latency?
Often, when retrieval quality limits answer quality. In Anthropic's study, adding a reranker moved the reduction in retrieval failures from 49% to 67%. The extra time grows with the shortlist size, so that size is a tuning choice.
Can an LLM be used as a reranker?
Yes, a general model can be prompted to rank candidates. Dedicated rerankers are usually cheaper and faster per query, which matters when every user request triggers a rerank.
Related
Hybrid search
Keyword ranking and vector similarity, fused into one result list.
Late interaction (ColBERT)
Many vectors per passage, matched token by token: finer retrieval, more storage.
Contextual retrieval
Give every chunk a line of context before it is indexed, so it can still be found.
Retrieval-augmented generation (RAG)
Fetch the right passages at query time, then answer from them with citations.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.