Suppose you build a fraud detector for a credit-card company. On a typical day, 99 out of every 100 transactions are honest and only 1 is fraud. You train a model, and it comes back 99% accurate — it gets 99 transactions right out of 100. That sounds brilliant, until you ask the one question that matters: how many frauds did it catch? The answer, for a model that simply predicts honest for everything, is zero. It is 99% accurate and completely useless. This is the central lie of accuracy, the most seductive metric in machine learning: a single number that can soar while the model does none of the work you actually hired it to do. In this lesson you will see exactly how accuracy betrays you under class imbalance, and why every practitioner reaches for sharper measures before trusting a number this high.
flowchart LR D["a day of 100 transactions"] --> H["99 are honest, class 0"] D --> F["1 is fraud, class 1"] style H fill:#2563eb,color:#fff style F fill:#b91c1c,color:#fff
What accuracy actually counts
Accuracy is the fraction of examples your model labels correctly: the number right, divided by the total. It is intuitive, it is a single number, and that is precisely why it is dangerous. When the classes are balanced — roughly equal numbers of each — accuracy is an honest summary of how often the model is right. The trouble begins when one class dwarfs the other, a situation called class imbalance. Then a model can rack up a high accuracy merely by ignoring the small class entirely and predicting the big one every time. The number looks wonderful; the behaviour is worthless. Accuracy rewards being right on average, but the examples you usually care about most are the rare ones.
def accuracy(preds, labels):
correct = 0
for p, y in zip(preds, labels):
if p == y:
correct += 1
return correct / len(labels)
labels = [0] * 99 + [1] # 99 honest, 1 fraud
always_honest = [0] * 100 # predict honest for everything
print('accuracy:', round(accuracy(always_honest, labels), 2))
print('frauds caught:', sum(1 for p, y in zip(always_honest, labels) if p == 1 and y == 1))
High accuracy, zero frauds caught
Run that example and watch the trap spring. The always-honest model scores 0.99 — ninety-nine correct predictions out of one hundred. By the only number it reports, it is nearly perfect. Yet it flags not a single fraud, because it never predicts fraud at all. The metric and the mission have come apart: accuracy measures agreement with the labels, and since almost every label is honest, agreeing with the majority is trivially easy. A model can be almost always right and still fail at the one job that justified building it. That gap — between how often it is right and whether it is right about the things that matter — is the whole reason accuracy cannot stand alone.
flowchart TD M["model predicts class 0<br/>for every transaction"] --> A["accuracy 0.99"] M --> C["frauds caught: 0"] A --> X["looks great on paper"] C --> Y["completely useless"] style A fill:#166534,color:#fff style Y fill:#b91c1c,color:#fff
def accuracy(preds, labels):
correct = 0
for p, y in zip(preds, labels):
if p == y:
correct += 1
return correct / len(labels)
labels = [0] * 99 + [1]
always_honest = [0] * 100
flagger = [0] * 97 + [1, 1, 1] # flags 3 transactions: 2 honest, 1 the real fraud
print('always-honest accuracy:', round(accuracy(always_honest, labels), 2))
print('flagger accuracy: ', round(accuracy(flagger, labels), 2))
print('flagger frauds caught: ', sum(1 for p, y in zip(flagger, labels) if p == 1 and y == 1))
The better model can score LOWER
Here is the most counter-intuitive part, and the reason accuracy alone cannot rank models. The flagger — a model that actually catches the one fraud — scores 0.98, slightly below the useless always-honest model. It trades two false alarms (honest transactions it wrongly froze) for the one fraud it stopped. If you picked models by accuracy alone, you would choose the version that lets every fraud through. This is not a contrived corner case; it is the normal state of fraud, spam, disease, and defect detection, where the interesting class is always the rare one. Any metric that lets a do-nothing model beat a working one is a metric you cannot trust to decide for you.
flowchart LR H["always-honest model<br/>accuracy 0.99"] --> Z["catches 0 frauds"] G["flagger model<br/>accuracy 0.98"] --> O["catches the fraud"] style H fill:#b45309,color:#fff style G fill:#166534,color:#fff
Two ways to be wrong
A classifier on a fraud problem can fail in two opposite directions. It can cry fraud when there is none — a false alarm that freezes an honest customer's card. Or it can stay quiet when there is fraud — a miss that lets the thief through. Accuracy lumps both failures into the same bucket of wrong, so a model with many misses and few false alarms can post the same accuracy as one with the opposite pattern. Those two failure modes carry completely different consequences, which is why folding them together hides exactly the information you need. Separating them — naming them, counting them, and weighing them by cost — is the project of every metric that comes after accuracy.
What does this print? Accuracy is correct predictions over the total. The model predicts 0 for every example.
def accuracy(preds, labels):
correct = sum(1 for p, y in zip(preds, labels) if p == y)
return correct / len(labels)
labels = [0, 0, 0, 1]
always_zero = [0, 0, 0, 0]
print(round(accuracy(always_zero, labels), 2))
Three of the four predictions match their labels.
3 divided by 4 is 0.75.
Write accuracy(preds, labels) returning the fraction of predictions that match their labels. preds and labels are equal-length lists.
def accuracy(preds, labels):
# correct divided by total
pass
Count agreements with sum and zip, then divide by len(labels).
Every prediction wrong gives 0.0; every one right gives 1.0.
def accuracy(preds, labels):
correct = sum(1 for p, y in zip(preds, labels) if p == y)
return correct / len(labels)
This accuracy only counts examples predicted as fraud AND truly fraud — it ignores every correctly-predicted honest transaction, so a model that predicts all-honest scores 0% instead of 99%. Fix the condition so it counts every agreement, not just the positive ones.
def accuracy(preds, labels):
correct = 0
for p, y in zip(preds, labels):
if p == 1 and y == 1:
correct += 1
return correct / len(labels)
Accuracy counts an example correct whenever the prediction equals the label.
Replace 'p == 1 and y == 1' with the simpler 'p == y'.
def accuracy(preds, labels):
correct = 0
for p, y in zip(preds, labels):
if p == y:
correct += 1
return correct / len(labels)
The laziest possible model always predicts the majority class. Write majority_baseline_accuracy(labels) returning the accuracy of that model — the size of the largest class divided by the total. This is the baseline every real classifier must beat, and it can be high purely from imbalance.
def majority_baseline_accuracy(labels):
# the biggest class count, divided by len(labels)
pass
Count the zeros; the ones are len(labels) minus that.
The majority is the larger of the two counts; divide it by the total.
def majority_baseline_accuracy(labels):
zeros = sum(1 for y in labels if y == 0)
ones = len(labels) - zeros
return max(zeros, ones) / len(labels)
A fraud dataset has 998 honest transactions and 2 fraudulent ones. Your model predicts 'honest' for all 1000. What is its accuracy, and what does that number hide?
It agrees with 998 of the 1000 labels, so the accuracy is 998/1000 = 99.8%. But it predicted 'honest' for both frauds, so it caught none of them. On imbalanced data a sky-high accuracy can coexist with a model that fails its actual job — which is exactly why accuracy alone cannot judge a classifier here.
Recap
- Accuracy is correct predictions over total — intuitive, but fragile.
- Under class imbalance (one class far larger), a model that always predicts the majority class scores high accuracy while ignoring the class you care about.
- A model that catches the rare class can score lower accuracy than one that ignores it, so accuracy cannot rank models on imbalanced problems.
- Ask whether one class dominates and whether the two mistakes cost differently; if so, do not trust accuracy on its own.
- The cure is to split errors by type — the confusion matrix (next lesson) and then precision and recall — instead of folding them into one number.
Next you will build the confusion matrix, the table that stops lumping every mistake together and makes the two kinds of error visible.