embedding quantization

Storing embedding values in fewer bits, such as one byte (int8) or one bit (binary) per dimension, to save memory and speed up search.

In more detail

Int8 is four times smaller than 32-bit floats and costs almost nothing in recall. Binary is 32 times smaller but coarser, so use it to find a fast shortlist and rescore that shortlist with fuller vectors.

Related terms

Taught in

References

Certification courses