What is vector quantization?
Vector quantization is the compression of embedding vectors into fewer bits per dimension, such as 8-bit, 4-bit or 1-bit codes, to cut the memory and storage a vector index needs, usually with a rescoring step to recover accuracy.
Checked
Embeddings are usually stored as 32-bit floats, and a vector index wants them in memory to search fast. At tens of millions of vectors that memory bill dominates the cost of running retrieval.
Quantization stores a compact code instead of the full vector. Search runs on the codes, and the top candidates are often rescored against the full-precision vectors kept on disk, which recovers most of the accuracy the compression cost.
The main kinds
Scalar quantization
Each 32-bit float becomes a smaller integer, commonly 8 bits. Simple, with modest accuracy loss.
Binary quantization
Each dimension becomes a single bit. The largest saving, so it is usually paired with oversampling and rescoring.
Product quantization
The vector is split into sub-vectors, and each is replaced by the ID of its nearest entry in a learned codebook.
Rotational quantization
The vector is rotated before quantizing so information spreads evenly across dimensions, which lets very low bit widths keep more accuracy. RaBitQ is a known example.
How engines ship it in 2026
Weaviate
1.40 (2026-10-14) made 4-bit rotational quantization generally available.
Qdrant
1.20 (2026-10-22) added TurboQuant 16-bit and 8-bit types. Qdrant also reports 21x compression on late-interaction page embeddings while keeping 96.5% of full-precision quality, in its own test.
Elasticsearch
Better Binary Quantization (BBQ), plus the disk-based DiskBBQ format. Elastic's docs note the recall trade-offs per setting.
Postgres
pgvectorscale adds statistical binary quantization. Pinecone's pg_pinecone runs an IVF index with RaBitQ, and Pinecone's own benchmark on 1M 768-dimension vectors reports a 136 MB index against 2.0 GB for a half-precision HNSW index.
How Motiontic's engineers apply it
Motiontic's engineers measure recall at the cut the application actually uses, such as top 10, on the client's own queries before and after quantization. Rescoring depth and oversampling are tuned against that number and the latency budget, because a vendor's headline compression ratio says little about a specific corpus.
Bits per dimension and compression compared with 32-bit floats
| Format | Bits per dimension | Compression vs float32 |
|---|---|---|
| Float32 (baseline) | 32 | 1x |
| Half precision | 16 | 2x |
| Scalar int8 | 8 | 4x |
| 4-bit | 4 | 8x |
| Binary | 1 | 32x |
Common questions
Does vector quantization reduce accuracy?
Some, and the loss grows as bits shrink. Most engines recover much of it by searching on compressed codes and then rescoring the top candidates with the full-precision vectors.
What is binary quantization?
Binary quantization stores one bit per dimension, a 32x reduction from 32-bit floats. It is fast and small, and is usually combined with oversampling and rescoring to keep recall acceptable.
Is quantization the same as using a smaller embedding model?
No. A smaller model produces different, usually shorter vectors. Quantization keeps the same model and stores each vector more compactly.
Related
HNSW
The layered graph index behind most vector search.
Vector database
Storage and nearest-neighbour search for embeddings, now in many shapes.
Late interaction (ColBERT)
Many vectors per passage, matched token by token: finer retrieval, more storage.
Embeddings
Meaning turned into vectors, so similar things can be found by distance.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.