A large language model is frozen the moment training ends. Everything it knows is whatever was in its training data on that date, so it has no idea your company rewrote the refund policy last Tuesday, and retraining the whole model for every document change would take weeks and cost a fortune. Retrieval-augmented generation (RAG) is the pattern that fixes this without touching the model's weights. Instead of hoping a fact was memorised, you fetch the relevant facts at query time and drop them into the prompt as context. The model then reads those facts and answers from them. RAG is the workhorse of production AI: every chat-with-your-documents feature, every support bot that quotes your help centre, and every coding assistant that reads your repository is some flavour of this one pipeline.
flowchart TD D["documents"] --> CH["chunk and embed"] CH --> IX["vector index<br/>built once, offline"] Q["user question"] --> E["embed the question"] E --> R["retrieve top-k chunks"] IX --> R R --> CTX["build context prompt"] CTX --> GEN["model generates answer"] style IX fill:#8b5cf6,color:#fff style GEN fill:#166534,color:#fff
Four named parts
The pipeline splits into two halves that run at different times. Offline, run once whenever documents change: split each document into chunks, pass every chunk through an embedding model to get a vector, and load them all into a vector index. That index is the searchable memory of the system. Online, run for every question: embed the question, retrieve the k most similar chunks, hand them to a context-builder that assembles a prompt, and let the generator — the language model — produce the answer. The four parts are the indexer, the retriever, the context-builder, and the generator. You have already built the first two in earlier lessons; this one bolts them into a single object and traces one query end to end.
const VOCAB = ["cat", "dog", "fish", "food", "water"];
function embed(text) {
const words = text.toLowerCase().split(/\W+/);
const v = new Array(VOCAB.length).fill(0);
for (const w of words) {
const i = VOCAB.indexOf(w);
if (i >= 0) v[i] += 1;
}
return v;
}
function dot(a, b) { let s = 0; for (let i = 0; i < a.length; i++) s += a[i] * b[i]; return s; }
function mag(v) { return Math.sqrt(dot(v, v)); }
function sim(a, b) { const m = mag(a) * mag(b); return m === 0 ? 0 : dot(a, b) / m; }
const store = [];
function index(text) { store.push({ text, vec: embed(text) }); return store.length - 1; }
function retrieve(queryVec, k) {
const scored = store.map((c, i) => ({ i, score: sim(queryVec, c.vec) }));
scored.sort((a, b) => b.score - a.score);
return scored.slice(0, k).map(s => s.i);
}
index("the cat drinks water");
index("a dog eats dog food");
index("fish live in water");
const q = embed("what does the cat drink");
console.log("top chunks:", retrieve(q, 2));
The testable boundary
The embedding in that example is a toy — it counts how often each vocabulary word appears, which is not how a real model represents meaning. But the pipeline around it does not care. Ranking chunks by cosine similarity, slicing to the top k, and formatting a prompt are all plain arithmetic you can run and assert on. The only stochastic step is the final generate, and that is precisely why these exercises leave it out. You build and test the entire retrieval pipeline with zero model calls, asserting on the context it assembles. When a real model is finally wired in at the end, it is the last, swappable component — not the load-bearing foundation.
flowchart LR Q["query"] --> E["embed"] E --> S["search the index"] S --> K["top-k chunk ids"] K --> B["build context"] B --> P["prompt with sources"] P -.->|"model generates"| A["answer"] style P fill:#b45309,color:#fff style A fill:#8b5cf6,color:#fff
Why the split exists
The two halves have wildly different cost profiles, and the split exists because of that. Indexing a million documents is slow, but you only pay it when the collection changes — once a night in a batch job, say — and afterward every query is cheap because it searches a ready-made index. Do the same work online instead, embedding every document on every question, and each query becomes a thousand times slower. The index is pre-computation: spend the time once, answer fast forever. This is also why a stale index is a real and common bug — if documents changed but the index was never rebuilt, the retriever faithfully returns the old chunks, and the model answers confidently from outdated facts.
function buildContext(texts) {
return texts.map((t, i) => `[${i + 1}] ${t}`);
}
const chunks = ["the cat drinks water", "a dog eats food", "fish live in water"];
const topIds = [0, 2]; // pretend the retriever returned these
const picked = topIds.map(i => chunks[i]); // look up the text for each id
console.log(buildContext(picked));
The context window caps everything
A model can only read so much text at once — its context window is a hard ceiling on prompt size. Retrieve forty chunks and cram them all in, and you either overflow the window or pay for tokens the model never uses. Production context-builders therefore cap the total: they take the top k by relevance and, if that still overshoots the budget, keep dropping the lowest-ranked chunk until it fits. Ranking is exactly what makes that safe, because the chunks you discard are the ones least likely to matter. The same number k that bounds your retrieval cost also bounds your prompt, which is why choosing it deliberately is a real engineering decision rather than a default to forget about.
flowchart TD RET["retrieve top-k chunks"] --> CHECK["does it fit the<br/>context window?"] CHECK -->|"yes"| KEEP["keep all k chunks"] CHECK -->|"no"| DROP["drop the lowest-ranked<br/>chunk, then re-check"] DROP --> CHECK style KEEP fill:#166534,color:#fff style DROP fill:#b45309,color:#fff
What the assembled prompt looks like
The assembled prompt has a predictable shape: the numbered context lines, then the question. Numbering each chunk is not decoration — it is what lets the model, and the citations in a later lesson, point back at which chunk a claim came from. Build that structure by hand once and you can see exactly what the model will see: no hidden state, no magic, just a string you assembled from ranked chunks. Every RAG system, however fancy, ultimately does this — fetch, rank, format, ask. The fanciness is all in doing each step faster or more accurately, never in inventing a fundamentally different shape.
What does this print? The context-builder labels each chunk with its 1-based position.
function buildContext(chunks) {
return chunks.map((c, i) => `[${i + 1}] ${c}`);
}
console.log(buildContext(["alpha", "beta"]));
map produces a new array, one labelled string per chunk.
console.log of an array prints its JSON form.
Write buildContext(chunks) that returns an array where each chunk text is prefixed by its 1-based position in square brackets, like [1] first.
function buildContext(chunks) {
// return an array of labelled strings
}
Use map, and remember its callback gives you both the value and its index.
The label is
[+ (i + 1) +]because positions are 1-based but indices are 0-based.
function buildContext(chunks) {
return chunks.map((c, i) => `[${i + 1}] ${c}`);
}
This retrieve is meant to return the top-k chunk texts, but it returns all of them sorted — ignoring k, which can overflow the context window. Fix it so only the top k come back.
function dot(a, b) { let s = 0; for (let i = 0; i < a.length; i++) s += a[i]*b[i]; return s; }
function mag(v) { return Math.sqrt(dot(v, v)); }
function sim(a, b) { const m = mag(a) * mag(b); return m === 0 ? 0 : dot(a, b) / m; }
function retrieve(queryVec, chunks, k) {
const scored = chunks.map((c, i) => ({ i, score: sim(queryVec, c.vec) }));
scored.sort((a, b) => b.score - a.score);
return scored.map(s => chunks[s.i].text);
}
After sorting, the best chunks are at the front of the scored array.
slice(0, k) keeps only the first k — cap before you map to text.
function dot(a, b) { let s = 0; for (let i = 0; i < a.length; i++) s += a[i]*b[i]; return s; }
function mag(v) { return Math.sqrt(dot(v, v)); }
function sim(a, b) { const m = mag(a) * mag(b); return m === 0 ? 0 : dot(a, b) / m; }
function retrieve(queryVec, chunks, k) {
const scored = chunks.map((c, i) => ({ i, score: sim(queryVec, c.vec) }));
scored.sort((a, b) => b.score - a.score);
return scored.slice(0, k).map(s => chunks[s.i].text);
}
Write makeRAG(docs, embed) that returns { query(question, k) }. It should embed the question, rank the docs by cosine similarity to it, and return a prompt string shaped Q: <question> on the first line followed by the top-k docs labelled [1], [2] and so on. The embed helper is provided for you.
function embed(text) {
const v = [0, 0, 0];
const letters = ["a", "b", "c"];
const lower = text.toLowerCase();
for (let i = 0; i < lower.length; i++) {
const idx = letters.indexOf(lower[i]);
if (idx >= 0) v[idx] += 1;
}
return v;
}
function makeRAG(docs, embed) {
// docs: array of strings. Return { query(question, k) } that builds the prompt.
}
Pre-compute each doc's vector once when makeRAG is called, so query only embeds the question.
Rank by cosine similarity, slice to k, then join the labelled lines with newlines under the Q: line.
function embed(text) {
const v = [0, 0, 0];
const letters = ["a", "b", "c"];
const lower = text.toLowerCase();
for (let i = 0; i < lower.length; i++) {
const idx = letters.indexOf(lower[i]);
if (idx >= 0) v[idx] += 1;
}
return v;
}
function makeRAG(docs, embed) {
const stored = docs.map(t => ({ text: t, vec: embed(t) }));
function dot(a, b) { let s = 0; for (let i = 0; i < a.length; i++) s += a[i] * b[i]; return s; }
function mag(v) { return Math.sqrt(dot(v, v)); }
function cosine(a, b) { const m = mag(a) * mag(b); return m === 0 ? 0 : dot(a, b) / m; }
return {
query(question, k) {
const q = embed(question);
const scored = stored.map((d, i) => ({ i, score: cosine(q, d.vec) }));
scored.sort((a, b) => b.score - a.score);
const top = scored.slice(0, k).map(s => stored[s.i].text);
return "Q: " + question + "\n" + top.map((t, i) => `[${i + 1}] ${t}`).join("\n");
}
};
}
Why retrieve documents at query time (RAG) instead of training the model on your documents once?
RAG keeps knowledge out of the weights: editing the index takes seconds, while every change would otherwise require an expensive re-train, and the retrieved chunks give each answer a citable source.
Recap
- A language model is frozen at training time, so private or fresh knowledge must be fetched, not memorised — that is the entire reason RAG exists.
- The pipeline has two halves: offline (chunk, embed, index — rebuilt when docs change) and online (embed query, retrieve, build context, generate — run per question).
- Four named parts — indexer, retriever, context-builder, generator — and everything up to the generator is deterministic and testable with no model calls.
- The context window caps the prompt, so a context-builder ranks by relevance and caps at
k, dropping the lowest-ranked chunks first. - Watch the two traps: duplicate chunks waste budget, and similarity is not the same as usefulness — which is exactly what reranking fixes next.