Embedding
Rewriting meaning as a set of numbers that marks a spot
- An embedding is a word or sentence's meaning rewritten as a set of numbers marking a spot. Similar meanings land near each other.
- Nobody chose the spots by hand. Training scanned a huge pile of text for which words kept turning up in similar places, and the layout fell out on its own.
- No single number carries a name. What matters is the distance and direction to other spots, not any one value.
- Because of embeddings, different words that mean the same thing can still be found together. That single fact reshaped search and recommendations.
- Not just words — sentences, documents, pictures, and sounds can all be given a spot the same way.
Contents
1The analogy
A trail signpost never lists just place names. It reads something like "spring 0.6, summit 2.4," with distances to a few fixed landmarks. With only those numbers you can tell whether two signposts sit close together or far apart, without ever unfolding a map — matching numbers mean matching spots. An embedding does exactly this for words: it tags each one with signpost numbers.
The numbers on a signpost have nothing to do with the place's name. "Boulder rest stop" carries no location information in its name, but the distance figures carry the whole location. Embeddings work the same way. They don't write down the shape of the letters — they write down where the meaning sits.
That's why two spots with completely different names can still be neighbors if their numbers are close. "Cell phone" and "smartphone" are neighbors like that.
2In detail
Similar meanings land close together
In the space an embedding builds, distance is closeness of meaning. "Apple" and "pear" sit near each other; "apple" and "subway" sit far apart. Fruits cluster in one region, vehicles in another.
Why this matters: it lets a computer catch the relationship between two words that don't share a single letter. Old-style search only found a document about a "refund" if you typed the word "refund." With embeddings, typing "I want my money back" surfaces the refund page too, because the two phrases land in nearby spots.
Closeness is usually measured by how much two spots point the same way when viewed from a common starting point — direction carries meaning better than raw distance does.
The layout formed on its own during training
Nobody wrote down "apple goes here, pear goes next to it." The method is simpler than that: scan a huge pile of text over and over, pulling words that show up in similar spots closer together.
When "peeled an apple" and "peeled a pear" keep turning up in similar contexts, the more training runs, the closer their spots are pulled. Words that almost never share a context get pushed apart instead. Repeat this pulling and pushing enough times, and words with similar meanings gather together on their own.
That also means an embedding absorbs whatever the training text was like. A bias that places a certain gendered term near a certain job title slips in through this same process.
No single number carries a name
An embedding usually holds a few hundred to a few thousand numbers. It would be convenient if the first number always meant "sweetness" and the second meant "size," but it doesn't work that way. Each position is an axis training happened to settle on, which makes it hard for a person to label.
Direction still carries meaning, though. A famous observation: start from "king," subtract the direction toward masculine qualities, add the direction toward feminine ones, and you land near "queen." A direction pointing from a country's name toward its capital shows up in similar form across many countries.
These relationships don't always line up perfectly. Many of the celebrated examples were cherry-picked, and the pattern breaks down for less common words.
The same word moves depending on context
Early methods gave each word exactly one spot. That meant the "bank" in "river bank" and the "bank" in "savings bank" shared the same position.
Current methods only produce a spot once you feed in the whole sentence. Because the surrounding words are read together, the same letters land in different spots depending on context. The "bank" in a finance story ends up far from the "bank" in a river story.
It's not just text that gets a spot
A single sentence, or a whole document, can be given a spot too. Feed in a long piece of writing and out comes a set of numbers that captures its gist. Store that ahead of time, and later you can find the document by meaning.
Pictures and sounds can share the same space as well. Train a photo and its caption to land near each other, and you can search for a photo using words, or find a caption that fits a photo.
3More precisely
An embedding maps a discrete item onto a point in a continuous vector space. The number of dimensions usually runs from a few hundred to a few thousand; a larger count captures finer distinctions but costs more storage and computation. That mapping isn't a hand-built dictionary of meaning — it falls out of a layer inside a model trained on some other task, so what an embedding captures depends heavily on what the model was trained to do in the first place.
Closeness is typically measured with cosine similarity, where a narrower angle between two vectors pushes the value toward 1. Search systems pull documents in descending order of that value. Tools that let you spin an embedding space around on screen first have to squash it down to two or three dimensions, which means the neighborhoods you see are a simplified sketch of the real, much higher-dimensional layout.
The analogy has a limit. A signpost's distance is a real, walkable length on the ground, checkable and tied to a fixed reference point. An embedding's numbers have no reference point anyone defined, and the raw values can't even be compared against a different model's numbers. Placing two different embedding models' outputs side by side and measuring the distance between them is meaningless — comparison only holds within the same model.
4Try it yourself
5Common misconceptions
It's easy to think reading an embedding's numbers reveals the meaning, but actually no single number carries a name — meaning only exists as distance from other embeddings.
It's easy to think the same word always gets the same embedding, but actually current methods read the surrounding context, so a different sentence puts the word in a different spot.
It's easy to think an embedding is a neutral set of numbers, but actually whatever bias sat in the training text carries straight through into the layout.
7One-line summary
In shortAn embedding rewrites meaning as numbers the way a signpost writes distance, so things that mean the same can be found even when the words are different.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02