Module 5 · Prompting & Context Engineering ⏱ 17 min

From Prompt Engineering to Context Engineering

By the end of this lesson you will be able to:
  • Explain why the 2026 skill is engineering the whole context payload, not wording a single prompt
  • Name the five ingredients of a payload and measure their token proportions
  • Diagnose a misbehaving app by inspecting payload composition before editing wording

In the playground you typed one prompt, the model answered, and it felt like the whole job was finding the right words. In production the model rarely sees just your words. By the time it runs, the text in front of it is a payload you assembled: the standing instructions, the conversation so far, the documents you retrieved, the definitions of the tools it may call, and the user's latest message, all concatenated into one stream. The 2024 skill was wording a prompt. The 2026 skill is engineering that payload — choosing what goes in, in what order, inside a budget you cannot exceed.

flowchart LR
  S["system<br/>instructions"] --> W["the context window"]
  M["memory"] --> W
  R["retrieved data"] --> W
  T["tool definitions"] --> W
  U["recent turns"] --> W
  style W fill:#8b5cf6,color:#fff
A modern call sends five ingredients, not one prompt — they all land in the same window.

The five parts of a payload

Most production payloads are built from the same five ingredients, and naming them precisely is what makes a failure report readable. System instructions are the standing rules — persona, format, refusals — sent on every call. Memory is facts carried across sessions. Retrieved data is the chunks a search step pulled back to ground the answer. Tool definitions are the schemas of functions the model may call. Recent turns are the live conversation. The model does not see five labelled boxes; it sees one flat stream of tokens. Engineering context means deciding how much of each ingredient that stream contains.

A payload is a list of parts. Classify each by the role it plays. Press Run.
const NAMES = { system: 'instructions', memory: 'memory', retrieved: 'retrieved data', tools: 'tool definitions', turns: 'recent turns' };
function categoryOf(role) {
  return NAMES[role] || 'unknown';
}
console.log(categoryOf('system'));
console.log(categoryOf('retrieved'));
console.log(categoryOf('turns'));

From wording to composition

Prompt engineering treats the instruction as the lever: rewrite it, the answer changes. Context engineering recognises that the instruction is only one ingredient competing for space. The retrieved data you attach can outweigh the instruction ten to one, and the model reads the concatenation, not your intent. So the levers that actually move production behaviour are selection (what to include), ordering (what the model reads first), and budgeting (what fits). A beautifully worded instruction buried under two thousand tokens of loosely relevant chunks behaves like no instruction at all.

flowchart TD
  P["one crafted prompt"] --> X["the model sees<br/>only your wording"]
  A["assembled payload:<br/>instructions + data + memory"] --> Y["the model sees<br/>the whole concatenation"]
  style P fill:#b45309,color:#fff
  style A fill:#166534,color:#fff
One crafted prompt sends only wording; an assembled payload sends a whole concatenation.

Order is not cosmetic

Because the model reads the stream in sequence, where an ingredient sits changes how much weight it carries. The standing instructions usually go first so they set the frame before any data arrives; the live user message goes last so it is the freshest thing in attention. Sandwiched between them, the retrieved chunks and older turns occupy the middle — the region models are reliably worst at using, the lost-in-the-middle effect. So a chunk's position is part of its value: a marginally relevant chunk placed last can outweigh a highly relevant one buried in the middle.

Measure what you send

You cannot engineer what you do not measure. The first habit is to total the tokens by ingredient and look at the proportions, because the proportions reveal the real problem. If retrieved data is eighty percent of the window and your instructions are five percent, no rewrite of the instructions will fix a model that is mostly reading noise. The fix is upstream of the prompt: retrieve less, retrieve more selectively, or compress. Treat the breakdown the way a performance engineer treats a flame graph — the biggest bar is where the work and the cost actually live.

Total the tokens by role. Watch which ingredient dominates. Run it.
function tokensByRole(parts) {
  const totals = {};
  for (const p of parts) totals[p.role] = (totals[p.role] || 0) + p.tokens;
  return totals;
}
const payload = [
  { role: 'system', tokens: 300 },
  { role: 'retrieved', tokens: 1200 },
  { role: 'retrieved', tokens: 800 },
  { role: 'turns', tokens: 200 }
];
console.log(tokensByRole(payload));
flowchart LR
  I["instructions<br/>small"] --> STACK["context window"]
  D["retrieved noise<br/>large"] --> STACK
  STACK --> OUT["instructions drowned out"]
  style D fill:#b91c1c,color:#fff
  style I fill:#1e7f3d,color:#fff
When retrieved noise dwarfs the instructions, no wording fix will reach the model.
The diagnostic: which role uses the most tokens? Run it.
function tokensByRole(parts) {
  const totals = {};
  for (const p of parts) totals[p.role] = (totals[p.role] || 0) + p.tokens;
  return totals;
}
function dominantRole(parts) {
  const totals = tokensByRole(parts);
  let best = null;
  for (const role in totals) {
    if (best === null || totals[role] > totals[best]) best = role;
  }
  return best;
}
const payload = [
  { role: 'system', tokens: 300 },
  { role: 'retrieved', tokens: 2000 },
  { role: 'turns', tokens: 200 }
];
console.log('dominant:', dominantRole(payload));

A single health number

When you want one number to track, use the instruction share: the instruction tokens divided by the total. It is not a target to maximise — instructions should be as short as they can be — but a floor to defend. When instruction share collapses toward zero as a conversation grows, the model has stopped reading your rules and started reading the accumulated noise, and behaviour drifts. Watch that ratio over the life of a session and you will catch drift before users do.

Exercise

What does this print? categoryOf maps a role name to the ingredient it represents. There are two console.log lines.

const NAMES = { system: 'instructions', memory: 'memory', retrieved: 'retrieved data', tools: 'tool definitions', turns: 'recent turns' };
function categoryOf(role) { return NAMES[role] || 'unknown'; }
console.log(categoryOf('retrieved'));
console.log(categoryOf('tools'));
Exercise

Write tokensByRole(parts) that takes an array of {role, tokens} objects and returns an object mapping each role to the total tokens used by that role. parts is an array like [{role: 'system', tokens: 100}].

function tokensByRole(parts) {
  // sum tokens per role into an object
}
Exercise

This dominantRole should return the role that uses the MOST tokens — the ingredient to investigate first. But the comparison is backwards, so it returns the role using the FEWEST tokens instead. Flip the one operator so the largest total wins.

function tokensByRole(parts) {
  const totals = {};
  for (const p of parts) totals[p.role] = (totals[p.role] || 0) + p.tokens;
  return totals;
}
function dominantRole(parts) {
  const totals = tokensByRole(parts);
  let best = null;
  for (const role in totals) {
    if (best === null || totals[role] < totals[best]) best = role;
  }
  return best;
}
Exercise

Write instructionShare(parts) returning the instruction share — the fraction of total tokens that come from parts whose role is 'system'. If there are no tokens at all, return 0.

function instructionShare(parts) {
  // system tokens / total tokens (0 if total is 0)
}
Exercise

In 2026, what does context engineering mean, as opposed to prompt engineering?

Recap

  • The 2026 skill is context engineering: designing the whole payload — instructions, memory, retrieved data, tools, recent turns — not wording one prompt.
  • The model sees a flat concatenation; your instruction is one ingredient competing with the others for attention.
  • The real levers are selection, ordering, and budgeting; order matters because of the lost-in-the-middle effect.
  • Measure the breakdown by ingredient before editing wording — the biggest ingredient is where behaviour and cost live.
  • Track instruction share as a health floor; when it collapses, the model has stopped reading your rules.

Next you will make the model return structure you can rely on, and then pack these ingredients into a fixed budget.

Checkpoint quiz

Why is polishing the instruction string often not enough to fix a misbehaving model app?

A diagnostic shows retrieved-data tokens are 80% of the window and instruction tokens are 5%. What is the honest conclusion?

Go deeper — technical resources