The last lesson ended on an uncomfortable fact: accuracy folds every mistake into one number, so a model that freezes a thousand honest cards and one that lets a thousand frauds through can post the same score. To judge a classifier you have to prise those mistakes apart. The tool for that is the confusion matrix, a two-by-two table that sorts every prediction into one of four buckets depending on what you predicted and what was actually true. It does not invent new information; it merely stops hiding the information accuracy blurred together. Once the four counts sit in front of you, questions accuracy could not answer — how many frauds did we miss, how many honest customers did we anger — become a matter of reading a cell. This lesson builds that table by hand and then asks the question accuracy never could: what does each cell actually cost?
flowchart TD M["confusion matrix<br/>2 predictions by 2 realities"] --> P1["predicted fraud"] M --> P0["predicted honest"] P1 --> TP["actual fraud: TP, caught"] P1 --> FP["actual honest: FP, false alarm"] P0 --> FN["actual fraud: FN, missed"] P0 --> TN["actual honest: TN, correct"] style TP fill:#166534,color:#fff style FP fill:#b45309,color:#fff style FN fill:#b91c1c,color:#fff style TN fill:#2563eb,color:#fff
Four cells, four outcomes
Fix a positive class — the thing you are hunting, say fraud, coded as 1. Every prediction lands in exactly one of four cells. A true positive (TP) is a fraud you correctly flagged. A true negative (TN) is an honest transaction you correctly left alone. The two interesting cells are the mistakes. A false positive (FP) is an honest transaction you wrongly flagged — a false alarm that freezes a good customer's card. A false negative (FN) is a real fraud you let through — a miss. Read the names as a sentence: the first word says whether you were right (true) or wrong (false), the second says what you predicted (positive or negative). That naming convention is why every metric downstream stays readable, so learn it now.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return tp, fp, fn, tn
labels = [1, 1, 0, 0, 1]
preds = [1, 0, 0, 0, 1]
tp, fp, fn, tn = confusion_matrix(preds, labels)
print('TP', tp, 'FP', fp, 'FN', fn, 'TN', tn)
Reading the table
The example above turns five predictions into four counts. Two true positives means the model caught two of the three real frauds. One false negative is the fraud it missed. Two true negatives are the honest transactions it left alone, and zero false positives means it never cried wolf. Notice that the four counts add back up to the total number of predictions — nothing is lost, nothing is double-counted. That is the matrix's whole promise: a complete accounting of where each prediction landed, with the two failure modes in separate cells instead of mashed into one. From these four numbers you can recover accuracy, but you can also ask the sharper questions accuracy refused to answer.
flowchart LR G["two kinds of mistake"] --> FP["FP, false alarm<br/>freezes an honest card"] G --> FN["FN, miss<br/>lets a fraud through"] FP --> CF["annoying, cheap"] FN --> CN["costly, dangerous"] style FP fill:#b45309,color:#fff style FN fill:#b91c1c,color:#fff
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return tp, fp, fn, tn
labels = [0] * 100 + [1, 1, 1] # 100 honest, 3 frauds
preds = [0] * 95 + [1] * 5 + [1, 1, 0] # 5 false alarms, 2 caught, 1 missed
tp, fp, fn, tn = confusion_matrix(preds, labels)
print('caught (TP):', tp)
print('false alarms (FP):', fp)
print('missed (FN):', fn)
print('correctly left alone (TN):', tn)
Each cell has its own price
Run the larger example and the four cells tell a story accuracy cannot. The model caught two frauds, missed one, raised five false alarms, and correctly left ninety-five alone. Two classifiers with identical accuracy could have wildly different cells: one might miss every fraud while annoying nobody, the other catch every fraud while freezing a dozen honest customers. Which is better depends entirely on what each cell costs. A false negative in fraud detection can mean a stolen life savings; a false positive might mean a five-minute phone call to unblock a card. The confusion matrix does not decide that trade-off for you — it finally lays out the numbers you need to decide it. In the next lesson you will compress pairs of these cells into precision and recall.
Rows and columns: a convention to nail down
Textbooks draw the matrix as a grid, and the arrangement matters for reading it. The common convention in scikit-learn puts predicted values on the rows and actual values on the columns, with the negative class first — so the top-left cell is TN, the bottom-right is TP, and the off-diagonal cells are the two kinds of error. Other fields flip the axes. Rather than memorise a layout, build the counts yourself with four clear if branches, as the code above does; then no diagram convention can trip you up. What matters is that each prediction lands in exactly one bucket, and that you know which bucket means which kind of mistake.
flowchart LR A["accuracy"] --> N["counts TP and TN as correct"] N --> D["lumps FP and FN together as wrong"] D --> W["hiding which mistake dominates"] style D fill:#b91c1c,color:#fff
What does this print? It returns the four counts as a tuple, in the order TP, FP, FN, TN.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return tp, fp, fn, tn
print(confusion_matrix([1, 0, 1], [1, 1, 0]))
Index 0: predict 1, true 1, so that is a TP.
Index 1: predict 0, true 1, so that is an FN. Index 2: predict 1, true 0, so that is an FP. TN stays 0.
Write confusion_matrix(preds, labels) returning a dict with keys 'tp', 'fp', 'fn', 'tn'. The positive class is 1.
def confusion_matrix(preds, labels):
# tally the four cells; positive class is 1
pass
Four branches: p==1 and y==1, p==1 and y==0, p==0 and y==1, and the else.
Return a dict literal with the four counts.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return {"tp": tp, "fp": fp, "fn": fn, "tn": tn}
This matrix has its last two counters swapped: a prediction of fraud on an honest transaction (a false alarm) is counted as a miss, and a real fraud the model stayed quiet on (a miss) is counted as a false alarm. Fix the two wrong counters so a false alarm lands in FP and a miss lands in FN.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 0 and y == 0:
tn += 1
elif p == 1 and y == 0:
fn += 1
else:
fp += 1
return {"tp": tp, "fp": fp, "fn": fn, "tn": tn}
A false alarm is p == 1 and y == 0; it belongs in FP.
A miss is p == 0 and y == 1; it belongs in FN. Swap the two counters on those branches.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 0 and y == 0:
tn += 1
elif p == 1 and y == 0:
fp += 1
else:
fn += 1
return {"tp": tp, "fp": fp, "fn": fn, "tn": tn}
Using the provided confusion_matrix, write expected_cost(preds, labels, cost_fn, cost_fp) returning the total cost: every miss weighted by cost_fn plus every false alarm weighted by cost_fp. Correct predictions cost nothing.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return {"tp": tp, "fp": fp, "fn": fn, "tn": tn}
def expected_cost(preds, labels, cost_fn, cost_fp):
# misses cost cost_fn each, false alarms cost cost_fp each
pass
Call confusion_matrix once and pull out the 'fn' and 'fp' counts.
Multiply each count by its cost and add them; TP and TN contribute zero.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return {"tp": tp, "fp": fp, "fn": fn, "tn": tn}
def expected_cost(preds, labels, cost_fn, cost_fp):
cm = confusion_matrix(preds, labels)
return cm["fn"] * cost_fn + cm["fp"] * cost_fp
A fraud model predicts fraud (1) on a transaction that is actually honest (0). Which cell does this land in, and what is its everyday name?
You predicted positive (fraud) but the truth was negative (honest), so it is a False Positive. The model cried fraud where there was none — a false alarm, like freezing an honest customer's card. The two mistake cells differ only in which side of the prediction was wrong, but their consequences are completely different.
Recap
- The confusion matrix sorts every prediction into four cells: TP (caught), TN (correctly left alone), FP (false alarm), and FN (miss).
- Read the names as a sentence: first word is right or wrong, second is what you predicted.
- The four counts add up to the total, so the matrix is a complete, lossless accounting that accuracy blurred together.
- Accuracy is just the two diagonal cells over the total — it lumps the two off-diagonal mistakes, hiding which kind of error dominates.
- The same matrix implies different decisions in different problems, because FP and FN cost different amounts; always weigh cells by cost.
Next you will combine these cells into precision and recall, the two metrics that finally let you rank models on an imbalanced problem.