Binary vector embeddings are so cool
November 13, 2024
Embeddings let you turn an arbitrary piece of text into a series of numbers that manage to represent the meaning of the content. These embedding vectors can range in dimension from 512 on the shorter end all the way up to 8192 or more. Typically each of the weights in the vector is represented as a 32-bit floating point number.LLMs use embeddings to represent input text, but embeddings can also be used on their own.Embeddings enable you to easily search for pieces of content that have similar meanings by finding the similarity (commonly the cosine similarity) between the points represented by the vectors.
Source: Binary vector embeddings are so cool | Evan Schwartz
Vector embeddings are at the heart of many generative AI and semantic search systems. This gives a succinct high level description then explores the technique of quantization, a way of reducing size (and increasing the performance) at the cost of accuracy. It turns out that cost can be far lower than would be reasonably anticipated.
