Matryoshka embeddings
Embeddings trained so that the first part of each vector is itself a useful smaller embedding, allowing safe truncation.
In more detail
Kusupati and colleagues introduced Matryoshka Representation Learning in 2022. Models trained this way, such as OpenAI's with its dimensions parameter, can be shortened to save space; truncating a model not trained for it loses much more recall.
Related terms
- embedding dimensions — The number of values in each embedding vector, fixed by the model, which drives storage size and search cost.
- embedding quantization — Storing embedding values in fewer bits, such as one byte (int8) or one bit (binary) per dimension, to save memory and speed up search.
Taught in
- Embedding Models — AI Engineering: LLM Apps, RAG & Agents
References
- Kusupati et al.: Matryoshka Representation Learning (arXiv 2205.13147)
- OpenAI: Vector embeddings guide (dimensions parameter)