Skip to content

Embedding

πŸ“ Short definition

Coordinate vector encoding the meaning of a token or document in mathematical space β€” nearby concepts are nearby in that space.

πŸ“– Detailed definition

After tokenisation, each token becomes a fixed-size vector (e.g. 4096 dimensions for Llama 3.1 8B). In that space, β€œKing” and β€œQueen” are close; β€œParis” and β€œFrance” too. That geometry lets the model reason by analogy.

Two distinct uses β€” do not conflate:

  • Internal LLM embeddings: vectors computed at each layer during Prefill/Decoding β€” they feed attention.
  • Search embeddings (RAG): produced by specialized models (e.g. nomic-embed-text, sentence-transformers) to index documents in a vector database.

πŸ’‘ Why it matters for on-prem AI

RAG quality depends heavily on the embedding model chosen. A poor embedding model yields poor retrieval regardless of downstream LLM quality.

⚠️ Common pitfalls

  • Internal LLM embeddings and RAG search embeddings are different models for different jobs.
  • Using an English-only embedding model on a French corpus strongly degrades quality.

πŸ“š Go deeper

  1. 🧠 The Journey of a Prompt (Step 2: from token to vector)
  2. 🧩 RAG & Agents (how search embeddings feed RAG)

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.