Last lesson you scored a candidate line by adding up its squared residuals. That number already does the essential job — it is small for good lines and large for bad ones — but it carries two awkward properties that make it hard to talk about. It grows with the size of your dataset, since a hundred points produce a bigger total than ten even when each error is identical, and it is measured in squared units, which no human ever says out loud. Mean squared error fixes the first problem, and its cousin root mean squared error fixes the second. Together they are the working vocabulary for regression wrongness, used everywhere from model evaluation to the very loss that gradient descent minimises.
flowchart LR E["errors:<br/>prediction - actual"] --> Q["square each error<br/>(removes the sign)"] Q --> A["average them<br/>divide by the count"] A --> M["one number: the MSE"] style M fill:#10b981,color:#fff
Mean squared error, defined
MSE is the average of the squared differences between each prediction and its target: MSE = mean((prediction - actual)^2). Take every residual, square it, then divide by the number of points. Squaring handles the sign so misses cannot cancel, and the mean handles the dataset size so a hundred identical errors score the same as one. The result is a single, comparable number.
Two predictions that are each off by 1 give an MSE of 1, while two predictions off by 2 give an MSE of 4, not 2. That faster-than-linear growth is the whole point: doubling the error more than doubles the penalty. A model trained to minimise MSE is therefore pushed hardest against its worst mistakes.
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
print('perfect:', round(mse([2, 4, 6], [2, 4, 6]), 2))
print('off by 1:', round(mse([2, 4, 6], [3, 4, 5]), 2))
print('off by 2:', round(mse([2, 4, 6], [4, 6, 8]), 2))
Why square the error?
Squaring does three jobs at once, and understanding all three is what stops MSE from feeling arbitrary. First, it makes every error positive, so a model that is too high on one point and too low on another cannot hide behind cancellation. Second, it is smooth and differentiable, which matters more than it sounds: a function you can differentiate is a function you can run gradient descent on. Absolute error has a sharp kink at zero where the slope flips, which makes its gradient behave badly. Third, squaring grows quickly, so large misses dominate the average.
Other choices exist. Mean absolute error (MAE) averages the unsquared gaps instead. It is more forgiving to outliers and easier to read, but it does not punish a single catastrophic prediction the way MSE does. Which metric is right depends on how much a few big misses should matter.
flowchart TD S["squared error"] --> J1["always positive<br/>no cancellation"] S --> J2["smooth and differentiable<br/>gradient descent works"] S --> J3["large misses dominate<br/>outlier sensitive"] style S fill:#3776ab,color:#fff
import math
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
def rmse(predictions, targets):
return math.sqrt(mse(predictions, targets))
preds = [198, 205, 210]
targets = [200, 200, 200]
print('MSE :', round(mse(preds, targets), 2), 'units squared')
print('RMSE:', round(rmse(preds, targets), 2), 'units')
Root mean squared error: back to real units
MSE's awkward feature is its units. If you predict house prices in dollars, the errors are in dollars but the squared errors are in dollars-squared, a quantity nobody can picture. Root mean squared error (RMSE) takes the square root of MSE, snapping back to the original units: RMSE = sqrt(MSE). An RMSE of 30,000 dollars means the model's mistakes are on the order of thirty thousand dollars, which is a number you can actually act on.
RMSE is still sensitive to outliers — the square root softens the squaring but does not undo it — so a single huge miss still inflates the value. That is a feature when outliers are exactly the failures you want to catch, and a bug when they are mere measurement noise. Reporting both MAE and RMSE is a common honesty check: if RMSE far exceeds MAE, a few large misses are dominating your error.
The baseline you must beat
Every regression model deserves the same humbling question: is it actually better than predicting the average and going home? A model that always outputs the mean of the targets makes no use of the features at all, yet its MSE has a name you already know — it is the variance of the targets. That gives you a floor to compare against. A useful model's MSE sits below the variance; a model whose MSE equals the variance has learned nothing the mean did not already tell you. The ratio of how far below the variance your model lands is the idea behind R-squared, the most common unit-free score for regression. You do not need R-squared yet — you need the habit of comparing any error number to the mean-predictor baseline before celebrating it.
flowchart LR D["errors:<br/>1, 1, 1, 1, 9"] --> MAE["MAE = mean of values<br/>= 2.6"] D --> MSE["MSE = mean of squares<br/>= 17"] MSE --> N["MSE flags the big miss<br/>that MAE plays down"] style N fill:#b91c1c,color:#fff
What does this print? MSE is the mean of the squared differences.
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
print(round(mse([0, 10], [5, 5]), 2))
First error: (0 - 5)^2 = 25. Second error: (10 - 5)^2 = 25.
Mean of 25 and 25 is 25.0.
Write mse(predictions, targets) returning the mean of the squared differences between the two equal-length lists.
def mse(predictions, targets):
# average of (p - t) squared
pass
Sum (p - t) ** 2 across the pairs with zip.
Divide the total by len(predictions).
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
This function averages the ABSOLUTE error instead of the squared error, so it computes MAE rather than MSE — the squaring is missing. Fix the one line so it squares each difference.
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += abs(p - t)
return total / len(predictions)
Replace abs(p - t) with (p - t) ** 2.
On [1, 3] vs [3, 1] the squared mean is 4, but the absolute mean is only 2.
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
Write rmse(predictions, targets) returning the root mean squared error — the square root of the MSE. The mse helper is provided for you; focus on the new step of taking the square root (which is what returns the score to the target's real units).
import math
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
def rmse(predictions, targets):
# sqrt of mse
pass
Call the provided mse, then take math.sqrt of the result.
sqrt(25) is 5, and sqrt(0) is 0.
import math
def mse(predictions, targets):
total = 0.0
for p, t in zip(predictions, targets):
total += (p - t) ** 2
return total / len(predictions)
def rmse(predictions, targets):
return math.sqrt(mse(predictions, targets))
You fit a model on prices in dollars and get MSE = 4,000,000. You re-express those same prices in thousands of dollars and recompute MSE on the identical predictions. What is the new value?
MSE is in squared units. Dividing every value by 1000 divides every error by 1000, and each squared error by 1,000,000, so 4,000,000 becomes 4. The model is unchanged; only the ruler moved — which is exactly why MSE is not comparable across unit scales.
Recap
- MSE is the mean of the squared errors: averaging makes it comparable across dataset sizes rather than rising as you add data.
- RMSE is the square root of MSE, returning the score to the target's original units so it reads like a typical miss.
- Squaring does three jobs: it removes sign, it stays smooth for gradient descent, and it punishes large misses hardest.
- MAE averages unsquared gaps instead — more forgiving of outliers, and a good honesty check beside RMSE.
- A model's MSE is only meaningful against the variance baseline; matching the variance means you have learned nothing the mean did not.
Next you will turn this loss into movement — building the gradient-descent update rule that walks a weight downhill to the minimum MSE.