You pasted the answer straight into the prompt — the exact sentence the model needed — and it still replied wrong. The fact was present; it simply was not used. Position, not presence, decides what a model leans on, and the worst seat in a long context is the middle.
This is the lost-in-the-middle effect: models draw on information near the start and the end of a long context more reliably than on information buried in the centre. It is not a quirk of one model. It is a robust pattern across long-context systems, first measured carefully in 2023 and reproduced widely since. In this lesson you build a small, deterministic model of that positional bias so you can watch a correct answer disappear — no model calls, just a weight curve and a retrieval rule.
flowchart LR P0["pos 0<br/>weight 1.0"] --> P1["pos 1<br/>0.5"] --> P2["pos 2<br/>0.0"] --> P3["pos 3<br/>0.5"] --> P4["pos 4<br/>1.0"] style P0 fill:#166534,color:#fff style P2 fill:#b91c1c,color:#fff style P4 fill:#166534,color:#fff
A toy model of positional attention
Real attention is learned and dense, but its average shape across a long context is simple: strong at the two edges, weak in the centre. We capture that with a positional attention weight for slot i out of n. The slot's distance to the nearer edge is min(i, n - 1 - i) — zero at both edges, largest in the middle. Divide that distance by its maximum possible value and subtract from 1, and you get a weight that is 1 at the edges and falls to 0 at the centre.
Print the weights for seven slots and out comes a clean U: 1.00, 0.67, 0.33, 0.00, 0.33, 0.67, 1.00. The weight is a multiplier on how much a slot counts — an edge slot counts fully, and the very same content placed in the middle counts for almost nothing.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
const n = 7;
for (let i = 0; i < n; i++) {
console.log(`pos ${i}: ${posAttention(i, n).toFixed(2)}`);
}
Relevance times position
Every passage has two independent properties: how well it matches the question (relevance) and where it sits (position). What the model effectively uses is the product — relevance * attention(position). That product is where the surprise lives.
Imagine the real answer has relevance 1.0, a perfect match, but it lands in the middle where attention is 0. Its effective score is 1.0 * 0 = 0. A weaker distractor with relevance only 0.4, parked at an edge where attention is 1.0, scores 0.4 * 1.0 = 0.4. The distractor wins, and the model returns the wrong passage. Not because the answer was missing — because of where it sat. Run the demo and the rule holds every time: the needle is found at either edge and lost in the middle.
flowchart TD N["needle<br/>relevance 1.0"] --> M["in the MIDDLE<br/>1.0 x 0 = 0"] D["distractor<br/>relevance 0.4"] --> E["at an EDGE<br/>0.4 x 1.0 = 0.4"] M --> L["needle loses"] E --> L style L fill:#b91c1c,color:#fff
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
function pickedIndex(relevances) {
const n = relevances.length;
let best = 0;
for (let i = 1; i < n; i++) {
if (relevances[i] * posAttention(i, n) > relevances[best] * posAttention(best, n)) best = i;
}
return best;
}
function needleFound(relevances, needle) {
return pickedIndex(relevances) === needle;
}
console.log(`needle in MIDDLE (idx 2): ${needleFound([0.4, 0.4, 1.0, 0.4, 0.4], 2)}`);
console.log(`needle at LEFT edge (idx 0): ${needleFound([1.0, 0.4, 0.4, 0.4, 0.4], 0)}`);
console.log(`needle at RIGHT edge (idx 4): ${needleFound([0.4, 0.4, 0.4, 0.4, 1.0], 4)}`);
Why the shape is a U
Two pressures carve it. Primacy: the opening tokens set the frame everything after is read against, so they carry outsized weight. Recency: the closing tokens sit closest to the moment the model actually generates its answer, so they weigh heavily too. The middle is far from both anchors and gets squeezed between them.
No real model is a perfectly symmetric textbook U — every architecture bends the curve, and a well-trained long-context model flattens the dip more than a weak one. But the durable lesson survives the details: edges are safe, the centre is risky, and the longer the context, the deeper and wider the dip in the middle tends to run.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
function effectiveScore(relevances, idx) {
return relevances[idx] * posAttention(idx, relevances.length);
}
const middle = [0.4, 0.4, 1.0, 0.4, 0.4];
const edge = [1.0, 0.4, 0.4, 0.4, 0.4];
console.log(`needle in middle, effective score: ${effectiveScore(middle, 2)}`);
console.log(`needle at edge, effective score: ${effectiveScore(edge, 0)}`);
flowchart TD P["good prompt layout"] --> S["START: task + question"] P --> MID["MIDDLE: bulk reference text"] P --> LAST["END: repeat the key instruction"] style S fill:#166534,color:#fff style LAST fill:#166534,color:#fff style MID fill:#b45309,color:#fff
The fix is layout
You cannot always shorten the context, but you can almost always choose where things go. Put the task, the question, and anything the model must act on at the start, and repeat the single most important instruction again at the end — bookend the prompt with what matters. Let the bulk reference text sit in the middle, where being attended less is acceptable because it is there to be searched, not memorised.
The same content, the same length, simply reordered, can take a needle from an effective score of 0 to 1. Reordering costs nothing and risks nothing, which makes it the cheapest accuracy gain in prompt engineering — and the first thing to reach for when a long prompt starts misbehaving.
What does this print? The centre slot gets the lowest positional weight. With n = 5, the centre is index 2.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
console.log(posAttention(2, 5));
Index 2 is the centre of a 5-slot context (indices 0 to 4).
The centre is as far from both edges as possible, so its weight is the minimum.
Write posAttention(i, n) returning a positional weight that is 1 at both edges (index 0 and index n - 1) and falls to 0 at the centre. Compute each slot's distance to the nearer edge, divide by the largest such distance, and subtract from 1.
function posAttention(i, n) {
// 1 at the edges, 0 at the centre
}
distFromEdge = Math.min(i, n - 1 - i) — how far this slot is from the nearer edge.
midDist is that distance at the centre; frac = distFromEdge / midDist; return 1 - frac.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
This posAttention is upside down — it returns 1 in the middle and 0 at the edges, the exact opposite of the lost-in-the-middle curve, so a centre needle looks maximally attended. Flip the one returned expression so edges score 1 and the centre scores 0.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return frac;
}
Right now the function returns frac, which peaks at the centre.
Edges and centre swap over if you return 1 minus frac.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
You must place one crucial instruction somewhere in a 20-passage context. At which position is it LEAST likely to be lost?
Lost-in-the-middle means the edges — primacy at the start, recency at the end — are attended most reliably and the centre least. Either edge is safe; the single worst seat is the middle.
Write isNeedleLost(relevances, needleIndex) returning true when the passage at needleIndex would NOT be picked — that is, when some other passage has a strictly higher effective score (its relevance times positional attention). posAttention is provided for you. The needle itself has the highest relevance, so losing it means position beat relevance.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
function isNeedleLost(relevances, needleIndex) {
// true if any other passage beats the needle's relevance * posAttention
}
Work out the needle's own effective score first.
Then loop over every other index; if any has a strictly greater effective score, the needle is lost.
function posAttention(i, n) {
const distFromEdge = Math.min(i, n - 1 - i);
const midDist = Math.floor((n - 1) / 2);
const frac = midDist === 0 ? 0 : distFromEdge / midDist;
return 1 - frac;
}
function isNeedleLost(relevances, needleIndex) {
const n = relevances.length;
const needleScore = relevances[needleIndex] * posAttention(needleIndex, n);
for (let i = 0; i < n; i++) {
if (i === needleIndex) continue;
if (relevances[i] * posAttention(i, n) > needleScore) return true;
}
return false;
}
Recap
- Lost-in-the-middle: information near the start and end of a long context is used more reliably than information in the middle.
- Model the bias with a positional attention weight —
1at the edges,0at the centre — a U-curve built from each slot's distance to the nearer edge. - What the model effectively uses is relevance × attention(position), so a perfectly relevant answer in the middle can score
0and lose to a weak edge distractor. - The U comes from primacy (first tokens frame the read) and recency (last tokens sit next to generation); longer contexts deepen the dip.
- The fix is layout: bookend what matters, bury the bulk, and retrieve one best passage to an edge — free reordering, large accuracy gain.
Next you will turn these positional instincts into a retrieval step that ranks and places passages deliberately.