Machine learning is mostly one question in disguise: given what you have already seen, predict a number you have not. How heavy will a package be? What will the temperature be tomorrow? What price will a house sell for? When the answer is a continuous value, the task is called regression, and the simplest tool for it is a straight line.
A line is the smallest model that can still learn from data. Two numbers define it entirely — a weight (the slope) and a bias (where the line crosses the axis) — and the model's prediction is prediction = w * x + b. The feature x is the input you already know, such as square footage; the prediction is the output you want, such as price. Every learner who later trains a neural network is, at heart, still tuning numbers to push a prediction closer to the truth.
flowchart LR X["feature x<br/>(sqft)"] --> M["times weight w<br/>plus bias b"] M --> P["prediction<br/>(price)"] style M fill:#10b981,color:#fff
The parts of the model
The formula has three moving pieces, and naming them precisely is what makes the rest of the course readable. The feature x is the input — something you measure before making the prediction. The weight w says how much the prediction rises when the feature rises by one. The bias b is the prediction the model would make if the feature were exactly zero, a baseline the weight then lifts or lowers.
Start with the even simpler case of a line through the origin, prediction = w * x, where the bias is zero. One number, w, controls everything. Pick w = 2 and the model predicts 2, 4, 6 for the inputs 1, 2, 3. The weight is the model's entire opinion about the data, compressed into a single value.
def predict(x, w, b):
return w * x + b
for x in [1, 2, 3, 4]:
print('x =', x, '-> predict', predict(x, 2, 1))
How wrong is a line?
A prediction is only useful if you can tell when it is bad. The gap between what the model predicted and what actually happened is the residual, computed as prediction - actual. A positive residual means the model overshot; a negative one means it undershot. Residuals are the raw material of every loss function in machine learning.
Here is the catch that motivates the whole field: you cannot judge a line by summing its residuals, because the overshoots and undershoots cancel. A terrible line that sits far too high on half the points and far too low on the other half can still sum to zero. To score a line honestly you need a measure of wrongness that is never fooled by sign — and the standard choice is to square each residual first. That single decision, squared error, is what the next lesson formalises.
flowchart LR P["a data point"] --> G["vertical gap<br/>to the line"] G --> R["residual =<br/>prediction - actual"] R --> S["square it so<br/>sign cannot cancel"] style R fill:#b45309,color:#fff
def predict(x, w, b):
return w * x + b
xs = [1, 2, 3, 4]
ys = [2, 4, 6, 8] # the true line is y = 2x
w = 1.5 # a too-shallow guess
for x, y in zip(xs, ys):
r = predict(x, w, 0) - y
print('x =', x, 'residual =', r, 'squared =', r * r)
Best fit means lowest error
Now you can state what a line of best fit actually is: among all the lines you could draw, it is the one whose total squared error is the smallest. That criterion is called least squares, and it is the definition every regression model inherits. The model never sees the right answer; it searches for the weights that minimise a number you can compute.
How does it search? One option is brute force — try candidate slopes, score each, and keep the best, as the final exercise does. That works for a single weight on a small grid, but it explodes once you have several features. The efficient route is to follow the slope of the error downhill — gradient descent — which the next two lessons build. Grasping that the goal is a minimum is what makes the method make sense.
A line versus a flat guess
Before you trust a line, ask whether it earns its slope. The simplest possible model ignores the feature entirely and always predicts the mean of the targets — a flat horizontal line, whose total squared error is just the spread (variance) of the data. A regression line is only worth keeping if it beats that flat guess; if the feature carries no real signal, the best slope lands near zero and the line collapses back toward the mean.
This is the seed of every goodness-of-fit measure: how much of the wrongness did using the feature actually remove? Squared error lets you answer that with one subtraction, and it is why mean squared error — not just error — became the default metric for regression. A model that cannot beat the mean has learned nothing.
flowchart TD T["try slope w = 0"] --> E0["score its<br/>total squared error"] T2["try slope w = 1"] --> E1["score it"] T3["try slope w = 2"] --> E2["score it"] E0 --> M["keep the slope<br/>with the lowest score"] E1 --> M E2 --> M style M fill:#166534,color:#fff
def predict(x, w, b):
return w * x + b
def total_error(xs, ys, w):
return sum((predict(x, w, 0) - y) ** 2 for x, y in zip(xs, ys))
xs = [1, 2, 3]
ys = [2, 4, 6]
best_w = None
best_err = None
for w in [0, 1, 2, 3]:
err = total_error(xs, ys, w)
print('slope', w, '-> total squared error', err)
if best_err is None or err < best_err:
best_err = err
best_w = w
print('best slope:', best_w)
What does this print? The model applies a weight, then adds a bias.
def predict(x, w, b):
return w * x + b
print(predict(3, 2, 1))
Substitute x = 3, w = 2, b = 1 into w * x + b.
That is 2 * 3 + 1.
Write predict(x, w, b) that returns the model's prediction for a single input using w * x + b.
def predict(x, w, b):
# weight times feature, then add bias
pass
Multiply the weight by the feature first.
Add the bias after the multiplication.
def predict(x, w, b):
return w * x + b
This prediction applies the bias to the feature BEFORE multiplying, which is wrong — the bias must be added after the weight has done its work. Fix the one line so the prediction follows w * x + b.
def predict(x, w, b):
return w * (x + b)
The bias is added to the product w * x, not pushed inside it.
Remove the parentheses around x + b.
def predict(x, w, b):
return w * x + b
Write residual(x, y, w, b) returning the residual prediction - actual for one data point, where the prediction uses w * x + b and y is the actual target.
def residual(x, y, w, b):
# prediction minus actual
pass
Compute the prediction w * x + b, then subtract the target y.
At x = 3, w = 2 the prediction is 6, so 6 - 5 leaves 1.
def residual(x, y, w, b):
return w * x + b - y
Write best_slope(xs, ys, candidates) that searches a list of candidate slopes (bias held at 0) and returns the one with the LOWEST total squared error. This is least squares by brute force — the idea gradient descent will soon do far faster.
def predict(x, w, b):
return w * x + b
def total_error(xs, ys, w):
# sum of (prediction - actual)^2 with bias 0
pass
def best_slope(xs, ys, candidates):
# return the candidate with the smallest total_error
pass
total_error sums (predict(x, w, 0) - y) squared over the data.
Track the lowest-error candidate seen so far and return it at the end.
def predict(x, w, b):
return w * x + b
def total_error(xs, ys, w):
return sum((predict(x, w, 0) - y) ** 2 for x, y in zip(xs, ys))
def best_slope(xs, ys, candidates):
best = candidates[0]
best_err = total_error(xs, ys, best)
for w in candidates[1:]:
err = total_error(xs, ys, w)
if err < best_err:
best_err = err
best = w
return best
Recap
- Regression predicts a continuous number; the simplest model is a straight line
prediction = w * x + b. - The weight sets the slope, the bias sets the baseline; the feature is the input you measure.
- A residual is
prediction - actual; summing raw residuals fails because overshoots and undershoots cancel. - Least squares defines the best line as the one with the lowest total squared error — the minimum that gradient descent will hunt for.
- A line only earns its slope if it beats the flat mean prediction, and it only behaves inside the range you trained on.
Next you will turn that squared-error idea into a real metric — mean squared error and its root — so you can measure wrongness precisely before learning to minimise it.