Embeddings Explained: How Semantic Search Actually Works

The One-Paragraph Answer

An embedding is a dense vector — a fixed-length list of numbers — that represents the meaning of a piece of text. The trick is the geometry: texts with similar meanings produce vectors that point in similar directions in a high-dimensional space. "Car maintenance" and "automobile repair" barely share a word, but their embeddings land close together, because the model learned that they mean similar things. Semantic search is simply: turn the query into a vector, then find the documents whose vectors are closest to it.

How Embeddings Are Made

Embeddings come from a transformer encoder — the same architecture family as ChatGPT, but optimized for understanding rather than generating. The process:

1. Tokenize. The text is split into tokens, and each token starts as a random-ish vector.
2. Contextualize. Self-attention layers let every token "look at" every other token, so each token's representation becomes context-aware — "bank" near "river" means something different than "bank" near "loan".
3. Pool. The final layer's token vectors are combined (mean pooling, or the CLS token) into one fixed-length vector: the embedding for the whole text.

The model is trained so that texts retrieved together or judged similar end up with nearby vectors — that's the entire goal of contrastive training on pairs like (query, relevant document). The result is a bi-encoder: query and documents are embedded independently, which is what makes corpus-wide search fast and precomputable. (The trade-off versus joint-scoring cross-encoders is covered in reranking in RAG.)

Why Vectors Capture Meaning

Here's the intuition: each dimension of the embedding space acts like a learned "dial" that responds to some aspect of meaning — grammatical role, domain, sentiment, topic. A model with 1024 dimensions has 1024 dials to represent what a text is about. Because the dials were tuned on billions of examples, similar texts reliably turn the same dials the same way.

This is why embeddings beat keyword search on paraphrase: keywords match surface form, embeddings match underlying meaning. It's also why they fail on rare exact strings — product codes, names, clause numbers — which is exactly why production systems combine embeddings with BM25 keyword retrieval rather than choosing one.

How Similarity Is Measured

Once texts are vectors, "how related are these two texts?" becomes a geometry question. The standard answer is cosine similarity — the cosine of the angle between two vectors:

cosine_similarity(a, b) = (a · b) / (|a| × |b|)

Identical directions score 1, perpendicular score 0, opposite directions score −1. Cosine ignores vector length, which is what you want — "refund policy" and "our complete refund policy for all products" should match even though one embedding is longer.

At scale, scanning every document is too slow, so vector stores use Approximate Nearest Neighbor (ANN) indexes — most commonly HNSW — which trade a small recall loss for search in milliseconds. That's the machinery inside ChromaDB, FAISS, Qdrant, and Weaviate; our ChromaDB vs FAISS comparison covers choosing the store.

Embedding Models in 2026

Embeddings Beyond Text

The idea generalizes: CLIP-style models embed images and text into a shared space (search "red car" finds photos of red cars), audio models embed sound, and code models embed functions. Anything that can be encoded as a vector can be searched semantically. Within RAG, the embedding step is where raw documents — PDFs, contracts, manuals — become a searchable knowledge base, which is exactly the step the 30-minute tutorial and Lawyer Assistant's pipeline both rely on.

Frequently Asked Questions (FAQ)

What is an embedding in simple terms?

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings get vectors that point in similar directions in a high-dimensional space, so "car maintenance" and "automobile repair" land near each other even though they share almost no words.

How many dimensions do embeddings have?

It depends on the model: nomic-embed-text uses 768, mxbai-embed-large and BGE-M3 use 1024, and OpenAI's text-embedding-3-large uses 3072. More dimensions can capture more nuance but cost more memory and compute — 768–1024 is the practical sweet spot for most RAG systems.

What is cosine similarity?

Cosine similarity measures the angle between two vectors, ignoring their length. It's the standard way to compare embeddings: identical directions score 1, perpendicular score 0, opposite score −1. Semantic search finds the documents whose embedding vectors have the highest cosine similarity to the query vector.

Which embedding model should I use?

For local RAG: BGE-M3 if you need multilingual and hybrid (dense + sparse) capability, mxbai-embed-large as the general-purpose default, or nomic-embed-text for a small model with an 8192-token context. For cloud, OpenAI's text-embedding-3 models are the standard benchmark.

Can embeddings work across languages?

Yes, with multilingual models. BGE-M3 supports 100+ languages in one model, and Qwen3-Embedding is particularly strong for Chinese. A cross-lingual embedding model maps "how do I cancel" (English) and its Spanish or Dari equivalent into nearby vectors, enabling cross-language retrieval.

Do embeddings work for images and audio too?

Yes. The same idea extends beyond text: CLIP-style models embed images and text into a shared space, and audio models embed sound. Any data that can be encoded as a vector can be searched semantically. In RAG pipelines, the embedding step is what turns raw documents into searchable knowledge.

Sources

• BGE-M3 — Hugging Face (BAAI)
• Denser.ai: Hybrid Search for RAG (embedding fundamentals + benchmarks)
• Ollama Embeddings for Local RAG (model comparison, April 2026)
• Embedding Model Selection Guide (April 2026)
• Best Embedding Model for RAG 2026 (Milvus)
• Embeddings Explained — Text to Vectors for Search (2026)