Metadata & Filtered Search

Introduction

Real RAG is almost never "search all my vectors." It's "search this user's documents, from this source, after this date, that they're allowed to see." That's filtered search — combining semantic similarity with metadata constraints — and it's where toy demos become real products.

It's also deceptively tricky. Bolting a metadata filter onto an ANN index (HNSW/IVF) is a genuine systems problem: do it the naive way and you get empty results, broken recall, or — worst of all — leaked data across users. This lesson shows the metadata model, the three filtering strategies and their tradeoffs, and the security rule you must never break.

In this lesson you'll learn:

What metadata is and what it unlocks
Pre-filter vs post-filter vs integrated — and why filtering is hard
The empty-results and HNSW-fragmentation gotchas
Multi-tenancy — filtering as access control (a top RAG security bug)

Metadata: The Structured Half of Your Vectors

Every vector you store should carry metadata — structured fields stored alongside it: {source, title, date, author, user_id, org_id, doc_type, tags, permissions}. The vector captures meaning; the metadata captures everything else you need to filter on. Together they turn a vector store into a real retrieval system.

What metadata enables, all of it essential in production:

Multi-tenancy — user_id / org_id so each user only sees their data (security — more below).
Recency — date > '2026-01-01' so you retrieve current info, not stale docs.
Source / type — only doc_type = 'policy', only this knowledge base, only PDFs.
Permissions / access level — only docs this role is cleared to read.
Categories / tags — scope retrieval to a topic, product, or language.

Store metadata at ingestion time, right next to the embedding. A vector with no metadata is a retrieval system you can't actually control.

Why Filtering Is Harder Than It Looks

"Just search, then filter" sounds trivial — but ANN indexes are built for pure vector search, so combining them with a filter forces a real tradeoff. Three strategies:

1. Post-filtering — vector-search first (get the top-N), then drop results that fail the filter. Simple and fast. The trap is the empty-results problem: if your filter is restrictive, the top-N the index returned might contain few or zero matching docs — so you hand back fewer than k, or nothing, even though matching docs exist deeper in the index. Result counts become unpredictable.

2. Pre-filtering — filter by metadata first, then vector-search only the survivors. It's accurate (you never even consider disallowed docs). But when the filter is very selective (say 1% of vectors survive), it disrupts the HNSW graph — the navigation links pass through filtered-out nodes, so the search paths fragment and recall drops (and broad filters can be slow to pre-scan).

3. Integrated / filter-aware ANN (the modern fix) — the index navigates the ANN structure while respecting the filter at every hop. Qdrant's filterable HNSW adds extra graph links so filtered search stays connected; Weaviate's ACORN strategy (default in recent versions) handles low filter-query correlation well. This gives you speed and correctness — no empties, no graph breakage.

An infographic titled 'Filtered Search — Combining Meaning with Metadata' with three strategy cards. POST-FILTER (search, then filter): step 1 vector-search everything to get a top-N, step 2 drop results that fail the filter; verdict: fast, but the top-N may contain few or zero matches, giving empty or short results. PRE-FILTER (filter, then search): step 1 filter by metadata first to get a subset, step 2 vector-search only the survivors; verdict: accurate, but a very selective filter fragments the HNSW graph, giving poor recall or slow search. INTEGRATED (filter-aware ANN, marked best): navigate the ANN index while respecting the filter, as in Qdrant filterable HNSW or Weaviate ACORN; verdict: the modern fix — speed plus correctness, no empties, no graph breakage. A red security note: filtering is access control — in a multi-tenant app every query must filter by tenant_id or user_id, enforced server-side, because a missing or client-controlled filter lets user A retrieve user B's documents, a serious data leak. A note: most vector DBs pick a strategy for you, but knowing pre versus post explains why you got 2 results instead of 10 (post-filter) and why filtered search is slow (selectivity). A bottom banner: real RAG isn't search all vectors, it's search the right ones — meaning times metadata, filtered correctly.

What This Means in Practice

The good news: most vector databases choose a strategy for you — you just pass a filter, and a mature DB (Qdrant, Weaviate, pgvector with a real WHERE) does the right thing. But understanding pre vs post is what lets you debug the weird behavior:

"Why did I ask for 10 results and get 2?" → post-filtering with a restrictive filter — the top-N didn't contain 10 matches. Fix: retrieve more candidates (raise the pre-filter N) or use a filter-aware index.
"Why is my filtered search slow / low-recall?" → high filter selectivity breaking naive HNSW. Fix: a DB with a filterable index (Qdrant/Weaviate), or pre-filter in SQL (pgvector).

Rules of thumb: for heavy metadata filtering, prefer a DB with native filter-aware indexing (Qdrant, Weaviate). In pgvector, filtering is a plain SQL WHERE and the planner handles it well at moderate scale. And always request enough candidates so a post-filter doesn't starve.

The Security Angle: Multi-Tenancy

This is the most important paragraph in the lesson. Filtering is not just a convenience feature — it's your access-control boundary.

In any multi-tenant app (a SaaS, a shared assistant), every retrieval query must filter by tenant_id / user_id so that user A can never retrieve user B's documents. Get this wrong and your RAG system becomes a data-leak machine: one user asks a question and gets back a chunk of someone else's private contract, medical note, or message.

Two non-negotiables (recall the defensive-prompting mindset):

Enforce the tenant/permission filter on the server, derived from the authenticated session — never trust the client to send the right user_id (an attacker will just send someone else's).
Treat it like authorization, because it is. Test it: confirm a query as user A can't surface user B's data, including via edge cases (shared docs, deleted users, admin overrides).

A missing or client-controlled tenant filter is one of the most common — and most damaging — bugs in real RAG systems. Make it impossible to forget by enforcing it in one server-side layer every query passes through.

Filtered Search in Code

pgvector — metadata filtering is just SQL WHERE, and the planner combines it with the ANN order-by:

SELECT content
FROM   docs
WHERE  org_id = $tenant_id        -- access control: from the SESSION, never the client
  AND  doc_type = 'policy'
  AND  created_at > '2026-01-01'  -- recency
ORDER BY embedding <=> $query_vec  -- semantic rank, WITHIN the filtered set
LIMIT 5;

Qdrant — a structured filter passed alongside the query vector (filter-aware HNSW):

from qdrant_client import QdrantClient, models
client = QdrantClient(url="http://localhost:6333")

client.query_points(
    collection_name="docs",
    query=query_vector,
    query_filter=models.Filter(must=[
        models.FieldCondition(key="org_id",   match=models.MatchValue(value=tenant_id)),
        models.FieldCondition(key="doc_type", match=models.MatchValue(value="policy")),
    ]),
    limit=5,
)
# Chroma equivalent:  col.query(query_texts=[...], n_results=5,
#                                where={"org_id": tenant_id, "doc_type": "policy"})

🧪 Try It Yourself

Diagnose the filter. For each symptom, name the cause and fix:

"I request 10 results with a tight doc_type filter and often get 1–2." → ?
"Filtered search over my 50M-vector HNSW index is slow and misses obvious matches when the filter is very narrow." → ?
"A user reported seeing a document from a different customer's account." → ?

→ 1: post-filtering + restrictive filter (top-N had few matches) — retrieve more candidates or use a filter-aware index. 2: high selectivity fragmenting HNSW — use a DB with filterable HNSW (Qdrant) or pre-filter in SQL. 3: 🚨 a missing/broken tenant filter — a critical data-leak bug; enforce tenant_id server-side on every query now. (If you only fix one thing from this lesson, it's #3.)

Interactive: a filtered-search lab. A 10-document store carries metadata (owner user, doc_type, relevance) and is queried for 'billing refund' requesting k=5 results. The user sets a tenant filter (You only / No filter) and a doc_type filter, then picks a strategy — post-filter, pre-filter, or filter-aware — and watches the results and empty slots. The three failure modes are felt directly: post-filtering the top-N returns fewer than k (the empty-results problem — 'asked for 5, got 2' — with a candidate-N slider that recovers the missing matches once raised); pre-filter and filter-aware return the correct top-k; and turning OFF the tenant filter leaks another customer's documents into the results (flagged in red). The lesson made tangible: post vs pre explains 'why 2 results instead of 10?', filter-aware ANN (Qdrant filterable HNSW, Weaviate ACORN) is the modern best-of-both, and filtering is access control that must be enforced server-side on every query. Illustrative relevance scores.

Mental-Model Corrections

"Filtering is trivial — just search then drop." That's post-filtering, and it silently returns fewer than k (or zero) results when the filter is restrictive.
"Pre-filtering is always safest." Accurate, but a very selective filter fragments the HNSW graph → poor recall. Use a filter-aware index for that case.
"The DB handles filtering, so I don't need to understand it." You do — to debug "why 2 results?" and "why slow?", and to pick a DB that filters well.
"The client can send the user_id." Never. Derive the tenant/permission filter server-side from the session — client-controlled filters are a data-leak hole.
"Metadata is optional." It's how you do recency, sources, permissions, and multi-tenancy — a vector with no metadata is uncontrollable retrieval.

Key Takeaways

Real RAG is filtered search: semantic similarity × metadata (user_id, date, source, doc_type, permissions). Store metadata next to every vector.
Three strategies: post-filter (search→filter; risks empty/short results), pre-filter (filter→search; fragments HNSW at high selectivity), integrated filter-aware ANN (Qdrant filterable HNSW, Weaviate ACORN) — the modern best-of-both.
Knowing pre vs post explains "why 2 results instead of 10?" and "why is filtered search slow?"
Filtering is access control: in multi-tenant apps, always filter by tenant server-side — a missing filter leaks other users' data (a top RAG security bug).
Most vector DBs filter for you; pick one with strong filtered-search (Qdrant/Weaviate) when filtering is heavy, or use SQL WHERE in pgvector.