Embeddings Beyond Search: Classification, Clustering & Dedup

Introduction

You just built semantic search. But here's the thing most people miss: the moment text becomes a vector, you've unlocked far more than search. An embedding is a reusable, general-purpose representation of meaning — and you can feed it into any classic ML or vector operation.

That's the superpower of this lesson: embed once, reuse for many jobs. The same vectors that power search also power classification, clustering, deduplication, recommendations, anomaly detection, and caching — each is just a different operation on the same vectors. Once you see this, you stop reaching for a bespoke model (or an expensive LLM call) for every text task.

In this lesson you'll learn to use embeddings for:

Classification — label text by its nearest labeled examples (few-shot, no fine-tuning)
Clustering — discover topics in unlabeled text
Deduplication — find semantic near-duplicates (even paraphrases)
And more — recommendations, anomaly detection, semantic caching

The Big Idea: Embeddings Turn Text Into Features

Here's the reframe. In classic machine learning, you spend ages feature-engineering text into numbers a model can use. An embedding model does that for you, for free: text in → a meaningful feature vector out.

And once text is a feature vector, fuzzy "text problems" become clean "geometry problems":

"Are these two things similar?" → distance between vectors.
"What group does this belong to?" → cluster it falls in.
"What category is this?" → nearest labeled vectors.
"Is this an outlier?" → far from everything.

You don't train a new model per task — you reuse one embedding model as a universal feature extractor, then run cheap, well-understood operations on top. That's why this is one of the highest-leverage tools in the whole toolkit.

An infographic titled 'Embeddings: One Representation, Many Jobs' with four panels, each a small scatter plot. CLASSIFY (by nearest labels): two coloured clusters and a new '?' point joined by a dashed line to its nearest neighbours — embed a few labeled examples per class, a new text takes its nearest neighbours' label (kNN), few-shot, no fine-tuning, often cheaper than an LLM. CLUSTER (discover topics): three faint coloured blobs of points — run k-means or HDBSCAN on unlabeled text to group similar items into themes you didn't know existed, then label or summarize each. DEDUP (find near-duplicates): scattered grey points with two red points almost overlapping inside a dashed circle marked with an approximately-equal sign — any pair with cosine above a threshold is a semantic duplicate, even paraphrases, cutting 20 to 50 percent of redundant training data and often improving quality. AND MORE (same trick): recommend (more like this), anomaly detection (an outlier far from every cluster), and semantic caching. A callout: once text is a vector it's a feature, so fuzzy text problems become clean geometry problems. A bottom banner: embed once, reuse for search, classify, cluster, dedup, recommend — one model, many jobs.

Classification — Label by Nearest Examples

Need to route support tickets into billing / technical / shipping, tag content, or detect intent? You don't need to fine-tune anything — or even call an LLM. Embed a few labeled examples per class, then classify a new text by its nearest neighbors (kNN) or its nearest class centroid.

This is few-shot classification with zero training: add a handful of examples and you have a classifier. For higher volume or accuracy, train a tiny logistic regression on top of the embeddings — still seconds to train, milliseconds to run.

import numpy as np
# embed() = the normalized-embedding helper from the semantic-search lesson

examples = {
    "billing":   ["I was charged twice", "refund my payment", "wrong amount on my invoice"],
    "technical": ["the app keeps crashing", "I get an error on login", "the page won't load"],
    "shipping":  ["where is my order", "my package hasn't arrived", "what's my tracking number?"],
}
labels, texts = [], []
for label, exs in examples.items():
    labels += [label] * len(exs); texts += exs
X = embed(texts)                       # (9, d), embed the labeled set ONCE

def classify(text, k=3):
    q = embed([text])[0]
    nn = np.argsort(-(X @ q))[:k]      # k nearest labeled examples (cosine)
    votes = [labels[i] for i in nn]
    return max(set(votes), key=votes.count)   # majority vote

print(classify("my card got billed for $99 twice"))   # -> billing

When to use this over an LLM: for high-volume, well-defined categories, embedding classification is cheaper, faster, and more consistent than prompting a model on every item — and it can't 'creatively' invent a new label. Reach for an LLM when the categories are nuanced or change often.

Clustering — Discover Structure in Unlabeled Text

Flip the problem: you have 10,000 support tickets and no labels, and you want to know what people are actually contacting you about. Embed them all and run a clustering algorithm — k-means (when you roughly know how many groups) or HDBSCAN (when you don't, and want it to find the natural number and ignore noise). Tickets with similar meaning land in the same cluster, surfacing themes you didn't know existed.

from sklearn.cluster import KMeans

V = embed(all_tickets)                 # (10000, d)
km = KMeans(n_clusters=8, n_init="auto").fit(V)

# tickets sharing a km.labels_ value are the same theme:
for c in range(8):
    members = [all_tickets[i] for i in range(len(all_tickets)) if km.labels_[i] == c]
    print(f"cluster {c}: {len(members)} tickets — e.g. {members[0][:60]}")
    # (a nice touch: ask an LLM to NAME each cluster from a few of its members)

This is the engine behind customer-feedback analysis, content organization, and exploratory analysis — turning a pile of unstructured text into a labeled map. A common pattern: cluster with embeddings (cheap), then use an LLM to name/summarize each cluster (a few calls, not 10,000).

Deduplication & Near-Duplicate Detection

Exact-match dedup (set(), hashing) only catches identical text. It completely misses paraphrases — "the app crashes on startup" and "it closes immediately when I open it" are duplicates in meaning but share no words. Semantic deduplication catches them: embed everything, and any pair with cosine above a threshold is a near-duplicate.

V = embed(items)                       # (N, d), normalized
sim = V @ V.T                          # all pairwise cosines in one matmul

dupes = [(i, j) for i in range(len(items)) for j in range(i + 1, len(items))
         if sim[i, j] > 0.92]          # tune the threshold on your data
# (at scale: cluster first with k-means, then compare only WITHIN each cluster)

Why it matters: cleaning training data (semantic dedup can cut a dataset by 20–50% while maintaining or improving model quality — redundant data wastes compute and skews the model), merging duplicate tickets/issues, and spam/plagiarism detection. The threshold is the dial: higher = stricter (only very-close pairs); tune it on a few labeled examples.

A Few More (Same Trick)

Every one of these is "an operation on the same vectors":

Recommendations — embed items (products, articles, songs); "more like this" is just nearest-neighbor on item vectors. Embed users by what they liked → personalized recommendations.
Anomaly / outlier detection — a point that's far from every cluster is unusual: novel support issues, fraud, off-topic content, drift.
Semantic caching — cache an LLM's answer keyed by the query's embedding; when a paraphrased version of the same question arrives, serve the cached answer (a big cost/latency win for chatbots — covered in the production container).
Reranking signals, diversity, near-dup filtering in search results — all vector ops.

The pattern is always the same: embed → do geometry. Internalize that and you'll spot embedding solutions everywhere.

🧪 Try It Yourself

Match the task to the operation. For each, which embedding job solves it — classify, cluster, dedup, recommend, or anomaly?

Auto-tag 50,000 incoming emails into 5 known departments. → ?
You have 20,000 product reviews and no idea what themes they cover. → ?
Your training set is full of paraphrased near-copies bloating it. → ?
Flag the one support ticket this week that's unlike anything you've seen. → ?

→ 1: classify (kNN on a few labeled examples per department). 2: cluster (k-means/HDBSCAN, then name the clusters with an LLM). 3: dedup (cosine > threshold). 4: anomaly (far from every cluster). Notice you'd reach for an LLM on none of them first — embeddings are cheaper and more consistent for all four.

Interactive: one embedded point cloud, five operations. A single 2-D scatter of ~13 support tickets (billing, technical, shipping clusters plus one off-topic outlier) is re-interpreted through five tabs without anything re-embedding. CLUSTER shows the three themes k-means/HDBSCAN discovers from unlabeled text; CLASSIFY marks one labeled example per class and labels a new ticket the user picks by its nearest labeled example (kNN, correct/incorrect shown); DEDUP links every pair above a similarity threshold the user drags — at a high bar only the true near-duplicate ('double charged' ≈ 'charged twice') links, and lowering it wrongly merges distinct tickets; ANOMALY highlights the ticket farthest from every cluster centroid (the off-topic GDPR question); and RECOMMEND lets the user click any ticket to see its nearest neighbours as the 'more like this' set. The lesson's thesis made tangible: an embedding is a reusable feature, so classification, clustering, deduplication, recommendation, and anomaly detection are all just geometry on the same vectors — embed once, reuse everywhere.

Mental-Model Corrections

"Embeddings are just for search." No — they're a reusable feature representation; search is one of many jobs (classify, cluster, dedup, recommend, detect).
"Use an LLM for classification/clustering." Often the wrong default — for high-volume, well-defined tasks, embeddings are cheaper, faster, more consistent, and can't hallucinate a label. Save the LLM for nuance.
"I need a different model per task." No — one embedding model feeds all of them. Embed once, reuse everywhere.
"Dedup means exact matches." Semantic dedup catches paraphrases (cosine > threshold) that exact-match misses entirely.
"Clustering needs labels." The opposite — clustering is unsupervised; it's how you discover labels/themes in unlabeled text.

Key Takeaways

An embedding is a reusable feature vector — once text is a vector, fuzzy text problems become geometry problems. Embed once, reuse for many jobs.
Classification: label by nearest labeled examples (kNN) or a tiny logistic regression — few-shot, no fine-tuning, cheaper than an LLM for high-volume categories.
Clustering: k-means / HDBSCAN on unlabeled text discovers themes (then name each cluster with an LLM).
Deduplication: cosine > threshold finds semantic near-duplicates (even paraphrases) — cuts 20–50% redundant data, often improving quality.
And more: recommendations (nearest-neighbor), anomaly detection (far from all), semantic caching (cache by query embedding). The pattern is always embed → do geometry.