Matryoshka embeddings

Embeddings trained so that the first part of each vector is itself a useful smaller embedding, allowing safe truncation.

In more detail

Kusupati and colleagues introduced Matryoshka Representation Learning in 2022. Models trained this way, such as OpenAI's with its dimensions parameter, can be shortened to save space; truncating a model not trained for it loses much more recall.

Related terms

Taught in

References

Certification courses