Technique Updated 2026-04
Embedding
Definition
An embedding is a numerical representation (vector) of text or data, capturing its semantic meaning.
See also in the glossary
V
Vector Database
A vector database stores embeddings for semantic search and RAG at scale.
R
RAG (Retrieval-Augmented Generation)
RAG is a technique that connects an LLM to external data sources to generate more accurate and up-to-date answers.
L
LLM (Large Language Model)
An LLM is an AI model trained on billions of texts, capable of understanding and generating human language.
N
NLP (Natural Language Processing)
NLP is the field of AI that enables machines to understand, interpret and generate human language.
Tools that use embedding
Frequently Asked Questions
What are embeddings used for?
They measure semantic similarity between texts. Two sentences with the same meaning will have close vectors. It's the foundation of RAG.
Are embeddings and tokens the same?
No. A token is raw text. An embedding is its translation into a numerical vector in mathematical space.
What is an embedding in simple words?
An embedding turns text, an image, or other data into a list of numbers (a vector) that captures its meaning. Items with similar meaning land close together in this vector space, so a machine can compare them mathematically. It's how AI systems understand that "car" and "automobile" are related even though the words differ.
What is a word embedding exactly?
A word embedding is a vector that represents a single word's meaning, learned from how it appears across large amounts of text. Early models like Word2Vec and GloVe produced one fixed vector per word. Modern systems use contextual embeddings, where the same word gets a different vector depending on the sentence around it.
What is an example of an embedding?
Semantic search is the classic example. Embed a query like "how to reset my password" and a help article into vectors, then retrieve the article whose vector is closest. The same mechanism powers RAG pipelines, product recommendations, and duplicate detection. Tools like Perplexity and NotebookLM rely on embeddings to find relevant passages before answering.
How is an embedding created?
A neural network trained on large datasets converts each input into a vector, typically a few hundred to a few thousand numbers. In practice you call an embedding model, such as OpenAI's text-embedding-3, Cohere Embed, or an open-source model from Hugging Face, and it returns the vector for any text you send. No manual feature engineering is required.
Where are embeddings stored?
Embeddings are usually kept in a vector database such as Pinecone, Weaviate, Qdrant, or pgvector. These systems index millions of vectors and run fast nearest-neighbor searches to return the most similar items in milliseconds. A vector database is the standard storage layer behind semantic search and RAG applications.
Can embeddings represent images and audio too?
Yes. The same principle applies to images, audio, and video. Multimodal models like CLIP place images and text in a shared vector space, so a text query can retrieve matching pictures. This is what enables "search by description" features and image similarity tools across modern AI products.
How do you measure similarity between two embeddings?
The most common metric is cosine similarity, which measures the angle between two vectors regardless of their length. A score near 1 means the items are nearly identical in meaning, while a score near 0 means they're unrelated. Euclidean distance and dot product are also used, depending on the model and the vector database.
