Your index is built and your similarity measure works — but over what? A whole book embedded into a single vector becomes one average point, and that point answers no specific question well. Retrieval needs the relevant passage, not the whole document, so every production pipeline chops documents into smaller pieces called chunks before embedding them. The chunk, not the document, is the atom of retrieval.
Here is the part that catches teams by surprise: where you cut is not a neutral preprocessing step. It quietly decides which query finds which passage. Cut in the middle of a sentence and you split one meaning in two; cut too small and a chunk loses the context that made it meaningful. This lesson builds three chunkers and shows them returning different results for the same query.
flowchart LR DOC["whole document"] --> SPLIT["split into chunks"] SPLIT --> C0["chunk 0 becomes vector 0"] SPLIT --> C1["chunk 1 becomes vector 1"] SPLIT --> C2["chunk 2 becomes vector 2"] style SPLIT fill:#8b5cf6,color:#fff
The chunk is the unit of retrieval
Once you chunk, each chunk is embedded to its own vector and added to the index as an independent item. A search no longer returns this document; it returns this passage, and the surrounding document is metadata you reattach afterwards. That means the chunk has to be self-contained enough to mean something on its own — a fragment with no subject, or a list item ripped away from its heading, will embed to a vector that matches the wrong things. The chunker's job is to produce pieces that are coherent units of meaning, not arbitrary slices of bytes.
Fixed-size chunking: the blunt instrument
The simplest chunker takes a fixed number of characters — or tokens — per chunk and strides through the text. It is fast, trivially simple, and entirely blind to structure: it will cheerfully slice a word, a sentence, or a heading in half. Look at the output below: the word Dogs is split across two chunks, and neither half is a real word an embedding model ever saw during training. Fixed chunking works when you do not care about boundaries, and it is the baseline every other strategy improves on.
function chunkByChars(text, size) {
const chunks = [];
for (let i = 0; i < text.length; i += size) {
chunks.push(text.slice(i, i + size));
}
return chunks;
}
const doc = "Cats are mammals. Dogs are mammals. Cars are vehicles.";
console.log(chunkByChars(doc, 20));
flowchart LR A["chunk ends with Do"] -->|"boundary cuts the word"| B["next chunk starts with gs"] style A fill:#b91c1c,color:#fff style B fill:#b91c1c,color:#fff
Chunking by sentence
A sentence is the smallest unit that usually carries a complete thought, so splitting on sentence boundaries respects meaning where fixed chunking does not. The chunker below groups a fixed number of sentences per chunk — two in the example — so each piece holds whole, readable statements. The trade is that chunks now vary in length, and a single very long sentence can still blow past a size budget you care about for embedding cost. Most production chunkers layer the two ideas: split into sentences first, then pack sentences up to a target size.
function chunkBySentences(text, perChunk) {
const sentences = (text.match(/[^.!?]+[.!?]/g) || []).map((s) => s.trim()).filter((s) => s.length > 0);
const chunks = [];
for (let i = 0; i < sentences.length; i += perChunk) {
chunks.push(sentences.slice(i, i + perChunk).join(" "));
}
return chunks;
}
const doc = "Cats are mammals. Dogs are mammals. Cars are vehicles. Boats are vehicles.";
console.log(chunkBySentences(doc, 2));
Overlap and the size sweet spot
Splitting on a clean boundary fixes the cut-mid-word problem but introduces a new one: an idea that spans the boundary now lives in two chunks, and a query about that idea may match neither of them strongly. Overlap fixes that by making consecutive chunks share a window of content — stride the window by less than its size, so the tail of one chunk becomes the head of the next. A boundary-crossing idea then appears, whole, in at least one chunk.
Chunk size itself has a sweet spot. Too small and a chunk like it failed has no referent for it — its embedding is near-meaningless. Too large and one chunk averages over many topics, diluting any single one and burying the matching passage inside noise.
flowchart LR W0["chunk A holds words 1 to 4"] -->|"overlap"| W1["chunk B holds words 3 to 6"] W1 -->|"overlap"| W2["chunk C holds words 5 to 8"] style W0 fill:#166534,color:#fff style W1 fill:#166534,color:#fff style W2 fill:#166534,color:#fff
function chunkWithOverlap(text, size, stride) {
const words = text.split(/\s+/);
const chunks = [];
for (let i = 0; i < words.length; i += stride) {
chunks.push(words.slice(i, i + size).join(" "));
if (i + size >= words.length) break;
}
return chunks;
}
const doc = "one two three four five six seven eight";
console.log(chunkWithOverlap(doc, 4, 2));
The same query, different answers
Change the chunker and you change the answers, even though the query and the document are identical. Fixed chunking might bury the decisive sentence half in one chunk and half in the next, so neither scores highly enough to surface. Sentence chunking keeps that sentence intact and lets it win. Overlap hedges the boundary, at the cost of storing the same words twice and slightly inflating the index. There is no chunker that is correct in general — only one that is correct for your documents, your queries, and the size of passages you want returned. That choice is made deliberately, and it is measured against real queries rather than assumed.
What does this print? The chunker strides through the text in steps of size.
function chunkByChars(text, size) {
const chunks = [];
for (let i = 0; i < text.length; i += size) chunks.push(text.slice(i, i + size));
return chunks;
}
console.log(chunkByChars("abcdef", 2));
Take 2 characters at a time from 'abcdef'.
ab, then cd, then ef — three chunks.
Write chunkByChars(text, size) that splits text into chunks of exactly size characters (the last chunk may be shorter), stepping by size each time.
function chunkByChars(text, size) {
// push slices of length `size`, stepping by `size`
}
Loop i from 0 to text.length, stepping by size.
Push text.slice(i, i + size) each iteration.
function chunkByChars(text, size) {
const chunks = [];
for (let i = 0; i < text.length; i += size) {
chunks.push(text.slice(i, i + size));
}
return chunks;
}
Write chunkBySentences(text, perChunk) that splits text into whole sentences (on ., !, or ?), then groups perChunk sentences per chunk, joined with a space. Return an array of chunk strings.
function chunkBySentences(text, perChunk) {
// split into sentences, then group perChunk per chunk
}
Match runs of non-terminal characters ending in a terminal mark to get whole sentences.
Then step through the sentences perChunk at a time and join each group with a space.
function chunkBySentences(text, perChunk) {
const sentences = (text.match(/[^.!?]+[.!?]/g) || []).map((s) => s.trim()).filter((s) => s.length > 0);
const chunks = [];
for (let i = 0; i < sentences.length; i += perChunk) {
chunks.push(sentences.slice(i, i + perChunk).join(" "));
}
return chunks;
}
This chunkWithOverlap is meant to slide a window of size words by stride each step, so edges overlap when stride is smaller than size. But the loop advances by size, so there is never any overlap. Fix the one expression that decides the step.
function chunkWithOverlap(text, size, stride) {
const words = text.split(/\s+/);
const chunks = [];
for (let i = 0; i < words.length; i += size) {
chunks.push(words.slice(i, i + size).join(" "));
if (i + size >= words.length) break;
}
return chunks;
}
Advancing by size means each window starts right after the previous one ended — no overlap.
The window should start stride words later, not size words later.
function chunkWithOverlap(text, size, stride) {
const words = text.split(/\s+/);
const chunks = [];
for (let i = 0; i < words.length; i += stride) {
chunks.push(words.slice(i, i + size).join(" "));
if (i + size >= words.length) break;
}
return chunks;
}
What problem does adding overlap between consecutive chunks most directly solve?
Overlap re-includes the words at each boundary, so an idea that would otherwise be sliced in two appears whole in at least one chunk and can still match a query strongly.
Recap
- Retrieval embeds chunks, not whole documents; the chunk is the unit a search returns.
- Fixed-size chunking is fast but blind to structure — it cuts words and sentences in half.
- Sentence-based chunking respects meaning by grouping whole sentences, at the cost of variable chunk length.
- Overlap (stride smaller than size) carries boundary-crossing ideas into at least one whole chunk.
- Chunk size has a sweet spot: too small loses context (the orphaned pronoun), too large dilutes the signal across many topics.
- The chunker is not neutral preprocessing — different chunkers return different top results for the same query.
Next you will meet the failure modes that survive all of this: the queries semantic search gets quietly, persistently wrong.