Embeddings explained: what they do and how to choose a model
Understand what embeddings do, see how text becomes a vector, and compare model options using your own search questions. Includes an API example, a simple diagram and an invoice example that keeps money and supplier IDs in exact database checks.
The question: What is an embedding, how do we make one, and which model should we choose for document search?
You’ll learn: Explain the search pipeline, read a vector comparison, understand provider options and plan a fair model test with clear privacy and upgrade rules.
For: Finance teams, product owners and engineers. No prior machine learning knowledge is needed.
Illustrative worked example · AI-assisted source check . This is not a human expert endorsement.
What is an embedding?#
Someone asks, “What do we do when only part of an order arrives?” The policy calls this a “partial goods receipt.” A search for the exact words may miss it. An embedding helps a search system find related meaning, even when the words differ. How semantic search works.
An embedding is a list of numbers produced by a model. That list represents a piece of content for tasks such as search. We also call it a vector. The system compares the question's vector with the vectors of saved passages, then returns nearby matches. Those matches still need checking.
It does not pull a policy out of the model's memory. You first supply your own approved policy text. It also does not confirm that an invoice is correct. “Paid” and “not paid” can share a topic while saying opposite things. A close match means “worth reading,” not “safe to approve.”
In our invoice example, use embeddings to find policy passages and similar descriptions. Use exact database fields for supplier IDs, purchase order numbers, money and currency. Supplier SUP-104 is not interchangeable with SUP-140, however similar their descriptions look.
How text becomes an embedding#
The usual text pipeline has three steps. A tokenizer breaks text into small pieces called tokens. A trained embedding model processes those pieces. Its output stage produces one fixed-length vector for the passage. Some model designs combine token representations through a step called pooling. The exact method depends on the model. Sentence Transformers model components.
You normally use an existing trained model. Sending a new policy to its embedding endpoint does not train it again. Ordinary chat output is different: asking a chatbot to invent a list of numbers does not give you a reliable embedding. Use the provider's embedding endpoint or a local embedding model.
Save passage vectors when documents change. Create a question vector when someone searches. Apply access and version rules before any passage reaches the user or answer model.
Select a box to inspect its role. On a small screen, scroll the diagram sideways.Keep the text, document ID, section and effective date.
Read the complete diagram as text
- Approved policy passage: Keep the text, document ID, section and effective date.
- Tokens → embedding model: Use the selected model and its document settings.
- Saved passage vector: Store the vector beside its source and access metadata.
- A person's question: For example: What if only part of the order arrived?
- Tokens → same model: Use compatible dimensions and the model's query settings.
- Question vector: Compare against permitted, applicable passage vectors.
- Read the matched source: Inspect the actual rule and its exceptions. Similarity does not approve payment.
- Approved policy passage → prepare text → Tokens → embedding model
- Tokens → embedding model → encode → Saved passage vector
- A person's question → prepare text → Tokens → same model
- Tokens → same model → encode → Question vector
- Saved passage vector → compare → Question vector
- Question vector → return candidates → Read the matched source
Keep both sides in the same model's search space. Use the same model version, output length and compatible preprocessing. Query and document settings may deliberately differ. For example, Cohere uses search_query for questions and search_document for passages. These are a matched pair, not interchangeable labels. Cohere search example.
Here is a real API pattern using invented, non-sensitive text. This example was not sent to an API. Running it requires the Python openai package, an approved API account and OPENAI_API_KEY stored on the server. It sends the input text to OpenAI and can incur charges. Embedding API reference.
from openai import OpenAI
client = OpenAI(timeout=30.0, max_retries=2)
model = "text-embedding-3-small"
dimensions = 1536
texts = [
"For a partial goods receipt, review the received quantity.",
"What if only part of the order arrived?",
]
response = client.embeddings.create(
model=model,
dimensions=dimensions,
input=texts,
encoding_format="float",
)
by_index = {item.index: item.embedding for item in response.data}
passage_vector = by_index[0]
question_vector = by_index[1]
assert len(passage_vector) == dimensions
assert len(question_vector) == dimensions
print("Passage dimensions:", len(passage_vector))This returns two vectors. It does not return a payment decision. In an application, save the passage vector with its source ID and model configuration. Reuse it until the text or configuration changes. Do not put API keys in browser code or log private input text.
What do the numbers mean?#
A vector's dimension is its number of values. A 1,536-dimensional vector contains 1,536 numbers. They are learned coordinates. You cannot usually name one coordinate “invoice risk” and another “supplier quality.” More dimensions also do not automatically mean better search.
To see the comparison, imagine these two-number vectors. We wrote these numbers by hand. They are not outputs from any model. The labels simply make the example easy to follow.
| Item | Invented vector |
|---|---|
| Question about a partial delivery | [1.0, 0.0] |
| Policy passage about a partial receipt | [0.9, 0.1] |
| Passage about office opening hours | [0.0, 1.0] |
One common comparison is cosine similarity. It measures how closely two vectors point in the same direction. For non-zero vectors, the formula is:
cosine(q, d) = sum(q[i] × d[i]) / (length(q) × length(d))
length(v) = sqrt(sum(v[i]²))
partial receipt: 0.9 / sqrt(0.82) ≈ 0.994
opening hours: 0 / 1 = 0The invented receipt passage wins this comparison. The number 0.994 is not a 99.4% chance that a decision is correct. Scores vary across models and datasets. Pick any acceptance threshold from your own tests. Semantic search commonly compares vectors this way. Sentence Transformers search guide.
Which models can produce embeddings?#
These are candidates to test, checked against official documentation on 14 September 2026. This is a shortlist, not a ranking. Model families that write answers and model families that produce search vectors can be different products.
| Model | Inputs and vector size | Useful candidate when |
|---|---|---|
OpenAI text-embedding-3-small | Text; 1,536 values by default; shorter output supported | You want a straightforward hosted text-search baseline. |
OpenAI text-embedding-3-large | Text; 3,072 values by default; shorter output supported | You can test whether its retrieval gains justify the extra cost. |
Cohere embed-v4.0 | Text and images; 256, 512, 1,024 or 1,536 values | Page appearance matters as well as extracted text. |
Google gemini-embedding-2 | Text, images, audio, video and PDF; 128–3,072 values | You need to search across different content types. |
BAAI bge-m3 | Multilingual text; 1,024-value dense vectors | You need a model you can operate on your own infrastructure. |
Specifications: OpenAI embeddings, Cohere Embed models, Google embeddings, BAAI model card.
Cohere's document and question settings must match their roles. Google's newer model uses task instructions in text instead of the older task_type field. Follow the exact model's example. A copied request for another model version can silently change what you are measuring. Cohere search settings, Google task instructions.
BGE-M3 also supports sparse and multi-vector retrieval. Its dense mode is the simplest comparison with the vectors above. Its published license is MIT. Self-hosting means your team operates the model, including updates, capacity and access controls. It does not remove operating costs. BAAI model card.
For the invoice case, start with extracted text and exact ERP records. Test page-image embeddings only if reading layout fixes a measured search problem. Finding a similar page still does not extract a dependable quantity or tax amount.
Our reference design starts with text-embedding-3-small for permitted policy text because the existing stack already uses it. That gives us a simple baseline to measure. It is not a finding that this model is best for invoices or multilingual policies.
Choose with your own questions#
Before trying models, ask finance reviewers to collect real search questions and mark the passages that answer each one. Include different languages, short descriptions, supplier abbreviations and poor scans. Add misleading examples: the same wording for another company, an expired policy and a passage that mentions a rule but does not contain its exception.
Split the questions into a development set and a held-out set. Use the first to adjust the setup. Keep the second unseen until you compare the finished candidates. Keep all questions about the same invoice or policy family in one split to reduce accidental leakage.
Give each candidate the same approved documents, chunks and permission rules. Also test a keyword-search baseline. Otherwise you cannot tell whether embeddings helped. Test the model's retrieval quality with exact vector search first, then measure any extra misses from a faster approximate index.
Recall@k asks: of the relevant passages we labelled, how many appeared in the first k results? If two passages are needed and the first five results contain only one, Recall@5 is 1/2, or 50%. Average that value across questions with at least one labelled answer. This is a search measure, not an invoice-resolution success rate. Retrieval evaluation tools.
Track false matches as well. For questions with no answer, measure how often the system presents an unrelated passage as sufficient evidence. Count wrong-company and wrong-version results separately. A high average recall cannot excuse either. Record search time and how much irrelevant text a reviewer must read.
Choose the least costly setup that meets your agreed quality and security targets. Include initial embedding, changed documents, search requests, storage, indexing, model hosting and reviewer time. This is our recommended evaluation plan; the numbers must come from your workload, not a vendor leaderboard.
Model changes and security#
Treat the model name, version, dimensions and text-preparation rules as one saved configuration. A new model normally needs a new set of document vectors. Equal vector lengths do not make two models compatible. Google explicitly requires re-embedding when moving from gemini-embedding-001 to gemini-embedding-2. Google migration guidance.
Build the new index beside the old one. Re-embed approved source text, repeat the held-out tests, then switch document and question processing together. Keep the previous configuration available for rollback. Revisit chunk boundaries if the new model handles different inputs; do not quietly mix incompatible vectors in one search.
Treat vectors as sensitive derived data. They are not encryption. Carry company, document, version and access metadata with every stored passage. Enforce current permissions before releasing results or sending them to an answer model. Remove searchable copies when a source is withdrawn. Keep restricted audit records without copying the sensitive passage into ordinary logs.
For a hosted model, check the actual service and account settings for processing location, retention and permitted data use. OpenAI's embedding endpoint lists no training use by default, but default abuse-monitoring retention still applies unless approved controls change it. Google's data terms distinguish paid and unpaid services, with regional exceptions. Neither should be read as “nothing is retained.” OpenAI data controls, Google API data terms.
The useful result is a search system that brings the right evidence to the reviewer. The invoice checks still use source records, exact calculations and the company's approval rules.