Skip to main content
CodePeetCodePeet
RAG & Retrieval Systems

Fixed, Recursive & Sentence Chunking

Introduction

Last lesson gave you the goal — one self-contained idea per chunk, because the chunk is the atomic unit of retrieval. This lesson gives you the first three tools to actually split text toward that goal, and they form a clean progression by how much document structure they respect:

  1. Fixed-size — respects nothing (just counts).
  2. Recursive — respects structure (paragraphs, then sentences, then words).
  3. Sentence — respects the sentence as the unit of meaning.

These are the bread-and-butter methods you'll reach for in 90% of real systems — and one of them, recursive, is the right default for most RAG you'll ever build. By the end you'll know exactly how each works, what it costs, and when to pick it — and you'll have watched them split text side by side.

In this lesson you'll learn:

  • Fixed-size chunking — the simple baseline, and why it butchers text
  • Recursive chunking — the separator hierarchy, and why it's the best default
  • Sentence chunking — splitting on meaning, and the abbreviation trap
  • The defaults to start from, and how to choose

The Spectrum: Structure-Blind → Structure-Aware

All three methods chase the same target (a coherent, single-idea chunk near a target size); they differ in how much of the document's structure they're willing to honor to get there. Picture them on a line from structure-blind to structure-aware — and notice that respecting structure is what keeps a chunk's idea intact. The interactive later in this lesson lets you flip between all three on the same document and see the difference.

An infographic titled 'Three Ways to Split Text — Fixed, Recursive & Sentence', showing three chunking methods by increasing respect for structure, from structure-blind to structure-aware. Method 1, Fixed-size: slice every N characters or tokens with overlap, ignoring sentences, paragraphs, even words. Fast, deterministic, no model, but it cuts mid-word and mid-sentence producing noisy, uninformative chunks; use for baselines, prototypes, and uniform data like logs and transcripts. Method 2, Recursive, marked BEST DEFAULT: try a hierarchy of separators, biggest first — paragraph, then line, then '. ', then space, then character — and if a piece is still too big, recurse to the next separator, keeping paragraphs then sentences then words whole. It gives roughly the structure of semantic chunking at roughly the cost of fixed-size, but separators must match the document type (prose is not code is not Markdown); use for docs, articles, books, and prose, which is most real-world RAG. Method 3, Sentence: split on sentence boundaries with an NLP library like spaCy or NLTK (not naive split on a period, which breaks on 'Dr.'), then pack sentences up to the size; it never breaks a sentence and gives clean units of meaning, but needs an NLP splitter and is weak on lists, tables, and code; use when sentence integrity matters and as a base for hierarchical or semantic chunking. A defaults strip says: start with Recursive, about 512 tokens (most do well at 200 to 500), 10 to 15 percent overlap, chunk by tokens not characters to match the embedder, and match separators to the document type. A banner reads: structure-awareness is nearly free and the payoff is huge — on structured docs, topic-aligned chunking hit 87 percent versus fixed-size's 13 percent; start Recursive.

1 · Fixed-Size Chunking

The simplest possible approach: count off N characters (or tokens) and cut, slide by N − overlap, repeat. It cares about length and nothing else — not sentences, not paragraphs, not even word boundaries.

import tiktoken
enc = tiktoken.get_encoding("cl100k_base")   # chunk by TOKENS, not characters

def fixed_chunks(text, size=512, overlap=64):
    toks = enc.encode(text)
    step = size - overlap                     # slide by (size − overlap)
    return [enc.decode(toks[i:i + size]) for i in range(0, len(toks), step)]

# Simple and fast — but it splits blindly:
#   "...refunds are processed within 5 busine|ss days..."   ← cut mid-word ✂️

Why it's tempting: it's fast, deterministic, and needs no model or NLP — you can chunk gigabytes in seconds.

Why it hurts retrieval: fixed boundaries slice straight through meaning. A cut lands mid-sentence — "...refunds are processed within 5 busine | ss days..." — and now neither chunk cleanly contains the fact. The embedding of a half-sentence captures noise, so the retriever misses the passage exactly when it's needed. On structured documents this is catastrophic (one clinical study measured 13% accuracy for fixed-size vs 87% for structure-aligned chunking).

One real upgrade for free: chunk by tokens, not characters (as above). Your embedding model has a token budget, and token-based sizing keeps you safely under it. But fixed-size remains a baseline — reach for it for prototyping or genuinely uniform data (logs, transcripts), then move up.

2 · Recursive Chunking — The Best Default

Recursive character splitting keeps fixed-size's speed but adds structural awareness, and it's the single best default for most RAG systems. The idea: instead of one blind cut size, give the splitter a hierarchy of separators ordered from most-meaningful to least, and let it work down the list.

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512, chunk_overlap=64,
    separators=["\n\n", "\n", ". ", " ", ""],   # ¶ → line → sentence → word → char
    length_function=len,                         # (use a token counter for token sizes)
)
chunks = splitter.split_text(text)

# How it works: try to split on "\n\n" (paragraphs). Any piece still > chunk_size?
# recurse and split THAT piece on "\n", then ". ", then " ", then "" (chars).
# → paragraphs stay whole when they fit; only oversized text is broken finer.

How the recursion works (this is the whole trick): start with the biggest separator, "\n\n" (paragraphs). Split on it. For any resulting piece that's still larger than chunk_size, recurse: split that piece on the next separator, "\n", then ". " (sentences), then " " (words), and only as an absolute last resort "" (characters).

The effect: paragraphs stay whole whenever they fit; sentences stay whole whenever they fit; words break only when a single word somehow exceeds the size. You get most of the coherence of expensive semantic chunking at almost the cost of fixed-size — which is why it's the default for documentation, articles, books, and prose (the bulk of real corpora).

The one thing you must tune: match the separators to the document type. Prose uses ["\n\n", "\n", ". ", " ", ""]; Markdown should split on headings (#, ##); code should split on functions/classes. LangChain ships from_language(...) variants for exactly this — using prose separators on code (or vice-versa) throws away the method's whole advantage.

3 · Sentence Chunking

Sentence chunking treats the sentence — the most fundamental unit of meaning in text — as sacred: split on sentence boundaries, then greedily pack whole sentences together up to the target size. A chunk is therefore always a clean set of complete sentences.

import spacy
nlp = spacy.load("en_core_web_sm")            # a real sentence segmenter, not text.split(".")

def sentence_chunks(text, size=512, overlap_sents=1):
    sents = [s.text.strip() for s in nlp(text).sents]
    chunks, cur = [], []
    for s in sents:
        if cur and len(" ".join(cur + [s])) > size:
            chunks.append(" ".join(cur))
            cur = cur[-overlap_sents:]          # carry the last sentence(s) over = overlap
        cur.append(s)
    if cur: chunks.append(" ".join(cur))
    return chunks
# Never splits a sentence — "Dr. Smith met Mr. Lee." stays one unit (naive .split('.') wouldn't).

The critical detail: use a real sentence segmenter, not text.split('.'). Naive period-splitting shatters on abbreviations ("Dr.", "U.S.", "v1.2"), decimals ("$4.50"), and ellipses — producing garbage 'sentences.' Libraries like spaCy and NLTK handle these correctly. Overlap here is natural and elegant: carry the last sentence (or two) into the next chunk.

When to choose it: when sentence integrity genuinely matters and your text is clean prose — and it's the standard building block for hierarchical and semantic chunking (next lesson), which decide where to group sentences by meaning. Its weakness: it's awkward on lists, tables, and code, where 'sentences' aren't the real units.

See Them Split (Interactive)

Theory lands when you watch it. Below, the same document is split three ways — flip the method and drag the chunk size and overlap. Start on Fixed-size at a small size and watch the ✂️ mid-word counter climb as chunks start and end in the middle of words; then switch to Recursive or Sentence and watch the breaks snap to real boundaries (the counter drops to zero and the "ends on a sentence" score jumps):

Pick a method and drag the size: Fixed-size butchers words mid-cut; Recursive & Sentence snap to real boundaries. Watch the ✂️ mid-word counter.

This is the lesson in one widget: fixed-size optimizes for a number; recursive and sentence optimize for meaning. That visible difference — clean breaks vs butchered words — is precisely what makes the difference between a chunk that retrieves and one that doesn't.

Defaults & How to Choose

You rarely need to agonize. Start here and tune on your data:

  • Method: start with Recursive — it's the best default for the majority of documents. Use Sentence when sentence integrity is paramount; keep Fixed-size for prototyping or uniform machine data.
  • Size: ~512 tokens is a strong starting point; most systems do well in 200–500 tokens. Smaller for fact lookup, larger for synthesis (recall the Goldilocks curve).
  • Overlap: 10–15% of chunk size — enough to rescue boundary-straddling ideas, not so much you bloat the index.
  • Unit: chunk by tokens, not characters, to match your embedding model's budget.
  • Separators: match them to the document type (prose ≠ Markdown ≠ code).

And the honest framing: these three are structural heuristics — they approximate "one idea per chunk" using layout (counts, separators, sentences) as a proxy for meaning. That proxy is cheap and gets you a long way. When layout isn't a good proxy for meaning — topic shifts mid-paragraph, ideas that span sections — you need methods that look at the meaning itself. That's the next lesson: semantic and sliding-window chunking.

🧪 Try It Yourself

Use the widget, then choose for real cases:

  1. In the interactive on Fixed-size, set size ≈ 100. How many chunks cut mid-word? Now switch to Recursive at the same size — what happens to that counter and the 'ends on a sentence' score?
  2. Which method for a company handbook of headings, paragraphs, and bullet lists — and what's the one parameter you'd customize?
  3. You inherit a pipeline using text.split('.') for sentence chunking, and chunks keep breaking inside "Dr. Lee" and "$4.50". What's the fix?
  4. You must chunk 5 million uniform log lines with a tight latency budget and no GPU. Which method, and why is it acceptable here?

→ (1) Fixed-size shows several ✂️ mid-word cuts; Recursive drops them to 0 and the sentence-end score jumps — it snapped to paragraph/sentence separators. (2) Recursive, and customize the separators to the doc's structure (split on Markdown headings, keep list items together). (3) Replace naive period-splitting with a real segmenter (spaCy/NLTK) that knows abbreviations and decimals. (4) Fixed-size (by tokens) — logs are short and uniform, structure barely matters, and you need raw speed with no model; this is exactly the baseline case where fixed-size is the right call.

Mental-Model Corrections

  • "Fixed-size is fine — it's simple." Simple, yes; but it cuts through words and sentences, producing noisy chunks that fail to retrieve. It's a baseline, not a destination.
  • "Recursive splits at a fixed size like fixed-size." No — it splits on a hierarchy of separators, breaking finer only when a piece is still too big. Paragraphs and sentences stay whole when they fit.
  • "Sentence chunking = split on every period." Naive .split('.') breaks on abbreviations and decimals. Use spaCy/NLTK.
  • "There's a universal best chunk size/separators." Size is query- and data-dependent (~512 to start); separators must match the document type.
  • "These methods understand meaning." They don't — they use structure (layout) as a proxy for meaning. Cheap and effective, but when layout ≠ meaning, you need semantic chunking (next).
  • "Chunk by characters." Chunk by tokens — that's the budget your embedding model and LLM actually count.

Key Takeaways

  • Three methods, increasing structure-awareness: Fixed-size (count & cut — fast but butchers text), Recursive (separator hierarchy ¶→line→sentence→word — the best default), Sentence (split on real sentence boundaries, pack to size).
  • Recursive keeps paragraphs/sentences whole by trying the biggest separator first and recursing finer only when needed — most of semantic chunking's quality at near fixed-size cost. Match separators to the doc type.
  • Sentence chunking needs a real segmenter (spaCy/NLTK) — never text.split('.') — and is the base for hierarchical/semantic methods.
  • Defaults: start Recursive, ~512 tokens (200–500), 10–15% overlap, chunk by tokens, separators matched to the document.
  • All three use structure as a proxy for meaning — cheap and strong, but blind to topic shifts within that structure.
  • Next: semantic & sliding-window chunking — splitting by meaning itself, for when layout stops being a good proxy.