85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What is late interaction (ColBERT)?

Late interaction is a retrieval method, introduced by the ColBERT model in 2020, that stores one vector per token of each document and scores a query by matching each query token against its closest document token.

Checked

A standard embedding squeezes a whole passage into one vector. That is compact, but fine detail gets averaged away. Late interaction keeps a vector for every token, so a query about one specific clause can match the tokens that carry it.

The idea comes from ColBERT, published by Omar Khattab and Matei Zaharia in April 2020. Queries and documents are encoded separately, so document vectors can still be computed ahead of time, and the interaction between them happens late, at scoring time.

How scoring works

For each query token, the system finds the most similar token vector in the document and keeps that maximum similarity. The document's score is the sum of those maxima across all query tokens, an operation usually called MaxSim.

The same approach extends to images. ColPali-style models treat patches of a page image as tokens, which lets retrieval run over scanned pages and slides without a text extraction step.

The storage trade-off

One vector per token means many vectors per passage, so storage and index size grow sharply. The industry answer has been compression and encoding. Qdrant reports 21x storage compression for Perplexity's late-interaction page embeddings while keeping 96.5% of full-precision quality without rescoring, a figure from Qdrant's own test.

Native support in 2026

  • Milvus 3.0

    Announced on 2026-07-16 with a StructList type for ColBERT and ColPali multi-vectors.

  • Weaviate 1.40

    Released on 2026-10-14 with multi-vector support on its disk-based HFresh index, encoded with MUVERA.

  • turbopuffer

    Its pricing page bills a late-interaction ANN query as two queries, a sign of the extra work involved.

How Motiontic's engineers apply it

Motiontic's engineers usually test late interaction first as a reranking stage over a shortlist from hybrid search, where storage cost stays small. It moves into first-stage retrieval only when evals on the client's documents, often dense technical or visual pages, show a gain that justifies the index size.

Single-vector embeddings compared with late interaction

Single-vector embeddings compared with late interaction
Single-vectorLate interaction
Vectors per passageOneOne per token or image patch
ScoringOne similarity per passageSum of per-token maximum similarities
Fine detailAveraged awayPreserved
StorageSmallMuch larger, reduced with quantization
Common roleFirst-stage retrievalReranking, or first stage for hard corpora

Common questions

What does late interaction mean in ColBERT?

It means the query and document are encoded separately and only compared at scoring time, token by token. That keeps document encoding offline while allowing finer matching than a single vector per passage.

Is late interaction better than a cross-encoder reranker?

They sit at different points. A cross-encoder reads query and passage together and is more precise but cannot precompute anything. Late interaction precomputes document vectors, so it can search larger sets, at a higher storage cost.

Which vector databases support multi-vector retrieval?

In 2026 Milvus 3.0, Weaviate 1.40 and Qdrant document native multi-vector support, and turbopuffer supports late-interaction queries at the price of two queries each.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.