The confusion matrix gave you four honest counts, but four numbers are awkward to compare. You want to ask two focused questions and get a single answer to each. The first: when this model raises the alarm, how often is it right? The second: of all the real frauds out there, how many does it catch? Those two questions have names. Precision answers the first, recall answers the second, and together they replace the one blunted number, accuracy, with two sharp ones. On an imbalanced problem — where the class you care about is rare — precision and recall are the metrics practitioners live by, because each isolates a different failure accuracy had quietly merged. This lesson implements both, then shows why you usually have to trade one against the other.
flowchart LR P["precision"] --> PQ["of those you flagged,<br/>how many were real?"] R["recall"] --> RQ["of the real frauds,<br/>how many did you catch?"] style P fill:#2563eb,color:#fff style R fill:#10b981,color:#fff
Precision: trust in an alarm
Precision asks, of every transaction the model flagged as fraud, what fraction truly was. The formula divides the frauds you caught by everything you flagged: TP / (TP + FP). A precision of 0.9 means nine in ten of your alarms were real — only one in ten froze an honest customer. When false alarms are the costly mistake, precision is the number you protect. A spam filter that bins a vital email hurts more than one that lets a piece of spam through, so you want high precision: when it says spam, it had better mean spam. Because the denominator counts everything you flagged, raising precision means flagging fewer honest transactions.
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return tp, fp, fn, tn
def precision(tp, fp, fn):
return tp / (tp + fp)
def recall(tp, fp, fn):
return tp / (tp + fn)
labels = [0] * 100 + [1, 1, 1]
preds = [0] * 95 + [1] * 5 + [1, 1, 0]
tp, fp, fn, tn = confusion_matrix(preds, labels)
print('precision:', round(precision(tp, fp, fn), 2))
print('recall: ', round(recall(tp, fp, fn), 2))
Recall: nothing slips past
Recall asks the opposite question: of all the real frauds, what fraction did you catch? The formula divides the frauds you caught by every fraud that existed: TP / (TP + FN). A recall of 0.67 means you stopped two out of every three frauds and let one through. When a missed fraud would be disastrous, recall is the number you chase. A cancer screening that misses a tumour is far worse than one that sends a healthy patient for a follow-up, so you want high recall: better to over-test than to let disease hide. Because the denominator counts every real fraud, raising recall means catching more of the frauds that are out there.
Why two numbers beat one
Precision and recall each hold the other kind of mistake accountable, which is exactly what accuracy refused to do. The fraud model above — two frauds caught, five false alarms, one missed — scores precision 0.29 and recall 0.67. Accuracy bundled all of that into a single ninety-something percent and called it a day. Split the question and the picture changes: a recall of 0.67 is honest about the one fraud in three that slipped past, and a precision of 0.29 is honest that most of your alarms are crying wolf. Neither number flatters the model, and that is the point. A metric that can look bad is a metric you can trust; a metric that always looks good is the one that gets you fired.
flowchart LR T["you cannot max both at once"] --> C["cautious model<br/>high precision, low recall"] T --> A["aggressive model<br/>low precision, high recall"] style C fill:#2563eb,color:#fff style A fill:#10b981,color:#fff
def confusion_matrix(preds, labels):
tp = fp = fn = tn = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
tp += 1
elif p == 1 and y == 0:
fp += 1
elif p == 0 and y == 1:
fn += 1
else:
tn += 1
return tp, fp, fn, tn
def precision(tp, fp, fn):
return tp / (tp + fp)
def recall(tp, fp, fn):
return tp / (tp + fn)
labels = [0] * 100 + [1, 1, 1] # 3 frauds among 103 transactions
cautious = [0] * 100 + [1, 0, 0] # flag only when very sure
aggressive = [1] * 10 + [0] * 90 + [1, 1, 1] # flag on any suspicion
for name, preds in [('cautious', cautious), ('aggressive', aggressive)]:
tp, fp, fn, tn = confusion_matrix(preds, labels)
print(name, 'precision', round(precision(tp, fp, fn), 2),
'recall', round(recall(tp, fp, fn), 2))
The trade-off you cannot escape
Run the two models and watch the see-saw move. The cautious model flags almost nothing, so every alarm it does raise is real — precision 1.0 — but it lets two-thirds of the frauds through. The aggressive model flags everything suspicious, catching every fraud — recall 1.0 — but more than three-quarters of its alarms are false. Push precision up and recall falls; push recall up and precision falls. This is not a flaw in your model but a fact about the problem: the boundary between the classes is fuzzy, and any threshold lands some honest cases on the fraud side or some frauds on the honest side. You do not get to maximise both. You choose which errors to tolerate based on what they cost.
flowchart TD Q["which matters more?"] --> M1["misses costly: maximise recall"] Q --> M2["false alarms costly: maximise precision"] Q --> M3["both equal: report F1"] style M1 fill:#b91c1c,color:#fff style M2 fill:#b45309,color:#fff style M3 fill:#3776ab,color:#fff
What does this print? Precision is TP divided by TP plus FP.
def precision(tp, fp, fn):
return tp / (tp + fp)
print(round(precision(3, 1, 2), 2))
The denominator is TP + FP = 3 + 1 = 4.
3 divided by 4 is 0.75.
Write precision(tp, fp, fn) and recall(tp, fp, fn). Precision is TP / (TP + FP); recall is TP / (TP + FN).
def precision(tp, fp, fn):
# TP / (TP + FP)
pass
def recall(tp, fp, fn):
# TP / (TP + FN)
pass
Precision divides tp by tp + fp; recall divides tp by tp + fn.
When fp is 0, precision is 1.0 no matter what fn is.
def precision(tp, fp, fn):
return tp / (tp + fp)
def recall(tp, fp, fn):
return tp / (tp + fn)
This precision divides TP by TP plus FN, which is actually the formula for recall — so a model that flagged nothing falsely would still score below 1. Precision must divide by the count of everything you flagged. Fix the denominator.
def precision(tp, fp, fn):
return tp / (tp + fn)
The denominator of precision counts everything flagged: TP + FP.
Replace (tp + fn) with (tp + fp).
def precision(tp, fp, fn):
return tp / (tp + fp)
Using the provided precision and recall, write f1(tp, fp, fn) returning the harmonic mean 2 * p * r / (p + r) of the two. F1 is high only when BOTH are high, which is why it punishes a model that is strong on one and weak on the other.
def precision(tp, fp, fn):
return tp / (tp + fp)
def recall(tp, fp, fn):
return tp / (tp + fn)
def f1(tp, fp, fn):
# 2 * p * r / (p + r)
pass
Compute p and r first, then return 2 * p * r / (p + r).
When both p and r are 1.0, F1 is 1.0.
def precision(tp, fp, fn):
return tp / (tp + fp)
def recall(tp, fp, fn):
return tp / (tp + fn)
def f1(tp, fp, fn):
p = precision(tp, fp, fn)
r = recall(tp, fp, fn)
return 2 * p * r / (p + r)
You run airport security screening. A false alarm means a passenger opens their bag for a quick check (mild cost); a miss means a weapon passes through (severe cost). Which metric should you maximise, even at the expense of the other?
A miss here is catastrophic and a false alarm is cheap, so you guard against misses by maximising recall — better to open a few extra bags than let a weapon through. Precision and F1 both balance away exactly the asymmetry that matters, so they are the wrong choice when the two errors cost very different amounts.
Recap
- Precision =
TP / (TP + FP): of those you flagged, how many were real — guard it when false alarms are expensive. - Recall =
TP / (TP + FN): of the real positives, how many you caught — guard it when a miss is catastrophic. - The two trade off: raising one usually lowers the other, because the class boundary is fuzzy and no threshold is free.
- F1, the harmonic mean of the two, is a fair summary only when the errors cost about the same; it can hide very different mistake patterns behind one number.
- Decide which error hurts more first, then choose the metric — and always report both precision and recall.
Next you will sweep the decision threshold to trace the full ROC and precision-recall curves, which show every precision-recall trade-off at once instead of picking a single point.