Similarity

An embedding gives every piece of text a set of coordinates on a map of meaning. Texts about similar things live on the same street; unrelated ones are across town. Draw an arrow from the centre of the map to each text's spot, and "how similar are these two texts?" becomes "how alike are these two arrows?".

There are three natural ways to compare two arrows:

• Cosine similarity compares the direction they point and ignores how long they are, like two people pointing at the same mountain, one with a longer arm.
• Euclidean distance is the straight-line distance between the two arrow tips, like measuring with a ruler.
• Dot product mixes both: pointing the same way and being long both make it bigger.

Three texts, each placed in a 3-dimensional space (real models use 384 to 3,072 dimensions, meaning numbers per vector; the arithmetic is the same):

a = (1, 2, 2), b = (2, 1, 2), c = (2, −1, −2)

1. Dot product: multiply position by position, then add. a·b = 1·2 + 2·1 + 2·2 = 8. a·c = 1·2 + 2·(−1) + 2·(−2) = −4.
2. Length (the norm, written ‖a‖): square each number, add, and take the square root. ‖a‖ = √(1 + 4 + 4) = 3, and b and c also have length 3.
3. Cosine: dot product divided by both lengths. cos(a, b) = 8 / (3·3) = 0.89. cos(a, c) = −4 / 9 = −0.44.
4. Euclidean distance: subtract, square, add, square-root. ‖a − b‖ = √(1 + 1 + 0) = 1.41. ‖a − c‖ = √(1 + 9 + 16) = 5.10.

Cosine runs from −1 (opposite directions) through 0 (unrelated, at right angles) to 1 (same direction).

flowchart LR
  A["a = (1, 2, 2)"] --> D["dot product<br/>a·b = 8"]
  B["b = (2, 1, 2)"] --> D
  A --> NA["length ‖a‖ = 3"]
  B --> NB["length ‖b‖ = 3"]
  D --> COS["cosine = 8 / (3 × 3) = 0.89"]
  NA --> COS
  NB --> COS
  A --> SUB["difference a − b = (−1, 1, 0)"]
  B --> SUB
  SUB --> L2["distance = √2 = 1.41"]

In code: worked_example computes every number in the table above from a, b and c, so you can check your hand arithmetic against it.

On the example: cos(a, b) = (1·2 + 2·1 + 2·2) / (3 · 3) = 8 / 9 = 0.89; ‖a − b‖ = √((1−2)² + (2−1)² + (2−2)²) = √2 = 1.41.

In Python:

import math
a, b = [1, 2, 2], [2, 1, 2]
# a · b = Σ a_i b_i
dot = sum(a_i * b_i for a_i, b_i in zip(a, b))
dot  # → 8
# ‖a‖
norm_a = math.sqrt(sum(a_i ** 2 for a_i in a))
# ‖b‖
norm_b = math.sqrt(sum(b_i ** 2 for b_i in b))
norm_a, norm_b  # → (3.0, 3.0)
# cos(a, b)
round(dot / (norm_a * norm_b), 2)  # → 0.89
# ‖a - b‖
round(math.sqrt(sum((a_i - b_i) ** 2 for a_i, b_i in zip(a, b))), 2)  # → 1.41

The code is cosine, dot and euclidean, each a line of NumPy.

In code: rank sorts a whole collection of documents from best to worst match under whichever of the three metrics you name.

Why it matters every vector database, semantic search box and RAG system does exactly this comparison, millions of times per second. Knowing which of the three a system uses, and why, is the difference between correct and silently wrong rankings.

Everyday picture trim every arrow to the same length, one unit. Now the only thing that can differ is direction, so "tips are close together" and "arrows point the same way" become the same statement.

Tiny example a = (1, 0) and b = (0.6, 0.8) both have length 1 (check: 0.6² + 0.8² = 0.36 + 0.64 = 1). Their cosine is 1·0.6 + 0·0.8 = 0.6. The squared distance is (1 − 0.6)² + (0 − 0.8)² = 0.16 + 0.64 = 0.8, and 2 − 2·0.6 = 0.8. Same number.

On the example: ‖a − b‖² = 1 + 1 − 2·0.6 = 0.8 = 2 − 2·0.6.

In Python:

# both length 1
a, b = [1, 0], [0.6, 0.8]
# for unit vectors, a · b is the cosine
cos = sum(a_i * b_i for a_i, b_i in zip(a, b))
# ‖a - b‖²
dist_sq = sum((a_i - b_i) ** 2 for a_i, b_i in zip(a, b))
# the two sides of the identity
round(dist_sq, 2), round(2 - 2 * cos, 2)  # → (0.8, 0.8)

L2-normalizing (dividing a vector by its own length) is what trims every arrow to length 1. After it, the dot product is the cosine, and distance is a decreasing function of the cosine. So once vectors are L2-normalized, cosine, dot product and Euclidean distance give the same ranking.

flowchart TD
  S{What was the model<br/>trained with?} -->|cosine| N[L2-normalize every vector<br/>once, at write time]
  S -->|dot product| D{Does the vector length<br/>carry signal?}
  D -->|yes, e.g. popularity| RAW[Keep raw vectors,<br/>rank by dot product]
  D -->|no| N
  N --> FAST[Rank by dot product:<br/>same order as cosine and L2,<br/>and the cheapest to compute]

In code: l2_normalize trims every row to length 1; rankings_agree_after_normalizing ranks random documents by all three metrics before and after normalizing and reports which rankings match, and l2_identity_check returns both sides of the identity for two random vectors.

Why it matters vector databases typically normalize on insert and then use the dot product, the cheapest of the three. If you mix normalized and unnormalized vectors, rankings quietly change. Use the metric the embedding model was trained with.

Everyday picture on the map of meaning, some shops are big and popular. A recommender may deliberately make popular items' arrows longer, so they win ties. Cosine throws that away; the dot product keeps it.

Tiny example (popularity_in_the_norm): the query points along (1, 0). A niche item (0.99, 0.10) is almost perfectly aligned. A popular item (0.95, 0.30) × 3 = (2.85, 0.90) is slightly off but three times longer.

Cosine prefers the niche item; the dot product prefers the popular one.

Why it matters some retrieval and recommender models are trained with the raw dot product and put useful signal in the length. Normalizing their vectors "to be safe" throws that signal away.

Everyday picture in a small village some neighbours live next door and some live across the valley. Now imagine a city with thousands of streets running in thousands of directions: pick random strangers and they all turn out to live about equally far from you. "Nearest" stops meaning much.

Tiny example scatter 1,000 random points in a cube and measure from a random spot. In 2 dimensions the nearest point is about 1/100th as far away as the farthest. In 1,000 dimensions it's about 90% as far: everyone is nearly equidistant.

On the example: by hand first: a query at (0, 0) and three points at (1, 0), (0, 2) and (3, 4) are 1, 2 and 5 away, so the ratio is 1 / 5 = 0.2. With 1,000 random points the demo table shows about 0.01 at d = 2 and about 0.9 at d = 1,000. The Python below draws its own random points, so it lands close by rather than on the same digits: 0.02 and 0.9.

In Python:

import math, random
def ratio(q, xs):
    # ‖q - x_j‖ for every j
    dists = [math.dist(q, x_j) for x_j in xs]
    # min over j ÷ max over j
    return min(dists) / max(dists)
ratio((0, 0), [(1, 0), (0, 2), (3, 4)])  # → 0.2
random.seed(0)
for d in (2, 1000):
    q = [random.random() for _ in range(d)]
    xs = [[random.random() for _ in range(d)] for _ in range(1000)]
    # one significant figure
    print(d, f"{ratio(q, xs):.1g}")  # → 2 0.02 1000 0.9

In code: curse_of_dimensionality scatters the random points, measures from a random query, and returns the nearest, farthest and ratio for each dimension.

Why it matters exact nearest-neighbour search in high dimensions gets slow and fragile, which is one reason vector search uses approximate indexes (primer.ml.embeddings.ann). Real embeddings aren't random: meaning puts them on a much lower-dimensional structure, so nearest-neighbour search still works well in practice.

Everyday picture a stadium crowd all facing the stage. Ask "are these two people facing the same way?" and the answer is "roughly yes" for everyone, so the question tells you little. Many embedding models behave like that crowd: their vectors bunch in a narrow cone, so even unrelated texts get cosines of 0.7 or more. This lopsidedness is called anisotropy ("not the same in every direction").

Tiny example build each vector as √3·m + n, where m is one direction everybody shares (the stage) and n is a unit-length direction of each text's own. For two unrelated texts the n parts are at right angles, so their dot product is (√3)² = 3 and each length is √(3 + 1) = 2, and the cosine is 3 / (2 · 2) = 0.75. For paraphrases, whose own parts agree with cosine ρ, the cosine becomes 0.75 + 0.25·ρ: about 0.79 to 0.86. All the signal is squeezed into a few hundredths.

On the example: unrelated, ρ = 0: (3 + 0) / (3 + 1) = 0.75. A paraphrase with ρ = 0.4: (3 + 0.4) / 4 = 0.85.

In Python:

import math
alpha, beta = math.sqrt(3), 1
# shared direction; two unrelated own directions
m, n1, n2 = (1, 0, 0), (0, 1, 0), (0, 0, 1)
# v = α m + β n
v1 = [alpha * m_i + beta * n_i for m_i, n_i in zip(m, n1)]
v2 = [alpha * m_i + beta * n_i for m_i, n_i in zip(m, n2)]
dot = sum(a * b for a, b in zip(v1, v2))
# the cosine, measured directly
round(dot / (math.hypot(*v1) * math.hypot(*v2)), 2)  # → 0.75
def cos_formula(rho):
    return (alpha ** 2 + beta ** 2 * rho) / (alpha ** 2 + beta ** 2)
# unrelated, and a paraphrase
round(cos_formula(0), 2), round(cos_formula(0.4), 2)  # → (0.75, 0.85)

In code: anisotropic_embeddings builds the pairs as α·m + β·n, pair_cosines scores each pair, and mean_center subtracts the average vector to remove the shared direction.

Picking a threshold from data, not folklore

Sometimes you need a yes/no cut-off: "are these duplicates?", "is this cached answer close enough?". Pick it by measuring. Label some pairs, score them, and choose the threshold with the best F1, the balance of precision (of the pairs I called similar, how many were?) and recall (of the truly similar pairs, how many did I catch?):

On a hand example: scores 0.9, 0.8, 0.7, 0.6 where the first two are true matches. Threshold 0.8 calls exactly those two matches: P = 2/2 = 1, R = 2/2 = 1, F₁ = 2·1·1 / 2 = 1. best_threshold finds 0.8.

In Python:

scores = [0.9, 0.8, 0.7, 0.6]
# the labels
is_match = [True, True, False, False]
# threshold 0.8
called = [s >= 0.8 for s in scores]
true_matches = sum(c and y for c, y in zip(called, is_match))
# precision
P = true_matches / sum(called)
# recall
R = true_matches / sum(is_match)
# F₁
2 * P * R / (P + R)  # → 1.0

flowchart LR
  L[Sample a few hundred pairs<br/>from your own data] --> H[Label each:<br/>same meaning or not]
  H --> E[Score every pair<br/>with THIS model]
  E --> W[Sweep thresholds,<br/>compute precision, recall, F1]
  W --> P[Pick the threshold that fits<br/>your cost of each error]
  P --> V[Store it with the<br/>model version]
  V -->|model upgraded| E

In code: f1_curve computes F1 at every candidate threshold for this plot; best_threshold returns the single best one with its precision and recall.

Why it matters

• Absolute scores mean different things for different models. "0.8" is not a universal "similar".
• Rank, don't threshold, whenever you can. A shared offset doesn't change which document is on top.
• When you must threshold (dedup, semantic caching, "no good answer found"), calibrate on labeled pairs for that model, and redo it when the model changes.

Answer each one out loud or on paper before you open it. If you can explain it, you know it.