What are Matryoshka embeddings?
Matryoshka embeddings are vectors trained so that their leading dimensions carry the most information, letting a system truncate a long embedding to a fraction of its dimensions and still get useful similarity search.
Checked
Index memory, storage and query time all grow with vector length. A 3,072-dimension embedding costs four times what a 768-dimension one does. Matryoshka training lets one model serve several sizes: the first 768 numbers of the long vector are themselves a usable embedding.
The method comes from Matryoshka Representation Learning by Aditya Kusupati and colleagues, first posted to arXiv on 2022-05-26. The authors report up to 14 times smaller embeddings at the same accuracy for ImageNet-1K classification and up to 14 times real-world speed-ups for large-scale retrieval.
How Matryoshka training works
During training, the loss is computed not only on the full vector but also on several nested prefixes of it, for example the first 64, 128, 256 and so on dimensions. The model learns to pack the most useful information first. Truncating then removes detail rather than meaning.
Models that support it in 2026
OpenAI text-embedding-3
Defaults to 1,536 dimensions for text-embedding-3-small and 3,072 for text-embedding-3-large, with a dimensions parameter to request shorter vectors. OpenAI's guide notes that vectors shortened by hand must be renormalised.
Google Gemini Embedding 2
Released in preview on 2026-03-10 with a 3,072-dimension default and Matryoshka training. Google recommends 1,536 and 768 as smaller sizes.
Voyage 4 family
Released 2026-01-15. Every model supports 2,048, 1,024, 512 and 256 dimensions, plus int8 and binary precision.
Why it matters in practice
Shorter vectors fit more of an index in memory and make every comparison cheaper. There is also a hard limit to clear: pgvector indexes the vector type up to 2,000 dimensions and halfvec up to 4,000, so a 3,072-dimension embedding needs halfvec or truncation before it can be indexed.
A common pattern is two-stage search: shortlist with short prefixes, then rescore the shortlist with the full vectors stored alongside.
Where it fails, and what it is confused with
Only for models trained this way
Truncating an embedding from a model without Matryoshka training discards information unevenly and can damage retrieval badly.
Quality still drops
Each cut loses some accuracy, and how much depends on the corpus. The loss has to be measured on real queries, not assumed from a vendor chart.
Versus quantization
Quantization stores each dimension in fewer bits; Matryoshka truncation keeps fewer dimensions. They stack, as Voyage 4's binary and short options show.
Versus PCA
PCA reduces dimensions after the fact with a separate projection. Matryoshka models are trained so a simple cut works without one.
How Motiontic's engineers apply it
Motiontic's engineers run the client's eval set at two or three dimension sizes before fixing a schema, and pick the smallest size that holds recall. Full-length vectors are kept where rescoring is planned, and the choice is recorded with the model version so a later re-embedding does not mix sizes in one index.
Ways to shrink embeddings compared
| Matryoshka truncation | Quantization | PCA | |
|---|---|---|---|
| What shrinks | Number of dimensions | Bits per dimension | Number of dimensions |
| Needs special training | Yes | No | No, but needs a fitted projection |
| Applied | At request or by slicing | At index time | After embedding |
| Combines with others | Yes, with quantization | Yes, with truncation | Rarely needed with Matryoshka models |
Common questions
Can any embedding be truncated to fewer dimensions?
Not safely. Only models trained with Matryoshka-style objectives keep their quality when cut. Others lose information unevenly.
Do OpenAI embeddings support Matryoshka truncation?
Yes. The text-embedding-3 models accept a dimensions parameter, and OpenAI's guide recommends it over manual slicing, which also requires renormalising the vector.
What is the difference between Matryoshka embeddings and quantization?
Matryoshka embeddings drop dimensions; quantization keeps every dimension but stores it with fewer bits. Both cut storage and can be used together.
Which dimension size should a RAG system use?
The smallest one that holds recall on the system's own eval set. Google, for example, recommends 3,072, 1,536 or 768 for Gemini Embedding 2, but the corpus decides.
Related
Embeddings
Meaning turned into vectors, so similar things can be found by distance.
Vector quantization
Fewer bits per dimension: smaller, cheaper indexes at a small cost in recall.
HNSW
The layered graph index behind most vector search.
Cosine similarity
The angle between two embeddings, and the default way to compare them.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.