Module 3 · Regression from Scratch ⏱ 17 min

The Idea of a Line of Best Fit

By the end of this lesson you will be able to:
  • Describe a linear model as a weighted feature plus a bias and compute its prediction
  • Explain the residual and why raw residuals cannot be summed to score a line
  • State least squares as the criterion that defines a line of best fit

Machine learning is mostly one question in disguise: given what you have already seen, predict a number you have not. How heavy will a package be? What will the temperature be tomorrow? What price will a house sell for? When the answer is a continuous value, the task is called regression, and the simplest tool for it is a straight line.

A line is the smallest model that can still learn from data. Two numbers define it entirely — a weight (the slope) and a bias (where the line crosses the axis) — and the model's prediction is prediction = w * x + b. The feature x is the input you already know, such as square footage; the prediction is the output you want, such as price. Every learner who later trains a neural network is, at heart, still tuning numbers to push a prediction closer to the truth.

flowchart LR
  X["feature x<br/>(sqft)"] --> M["times weight w<br/>plus bias b"]
  M --> P["prediction<br/>(price)"]
  style M fill:#10b981,color:#fff
A linear model: the feature goes in, a weight and a bias shape it, a prediction comes out.

The parts of the model

The formula has three moving pieces, and naming them precisely is what makes the rest of the course readable. The feature x is the input — something you measure before making the prediction. The weight w says how much the prediction rises when the feature rises by one. The bias b is the prediction the model would make if the feature were exactly zero, a baseline the weight then lifts or lowers.

Start with the even simpler case of a line through the origin, prediction = w * x, where the bias is zero. One number, w, controls everything. Pick w = 2 and the model predicts 2, 4, 6 for the inputs 1, 2, 3. The weight is the model's entire opinion about the data, compressed into a single value.

Predictions from one weight. Change w and the whole line tilts. Press Run.
def predict(x, w, b):
    return w * x + b

for x in [1, 2, 3, 4]:
    print('x =', x, '-> predict', predict(x, 2, 1))

How wrong is a line?

A prediction is only useful if you can tell when it is bad. The gap between what the model predicted and what actually happened is the residual, computed as prediction - actual. A positive residual means the model overshot; a negative one means it undershot. Residuals are the raw material of every loss function in machine learning.

Here is the catch that motivates the whole field: you cannot judge a line by summing its residuals, because the overshoots and undershoots cancel. A terrible line that sits far too high on half the points and far too low on the other half can still sum to zero. To score a line honestly you need a measure of wrongness that is never fooled by sign — and the standard choice is to square each residual first. That single decision, squared error, is what the next lesson formalises.

flowchart LR
  P["a data point"] --> G["vertical gap<br/>to the line"]
  G --> R["residual =<br/>prediction - actual"]
  R --> S["square it so<br/>sign cannot cancel"]
  style R fill:#b45309,color:#fff
A residual is the vertical gap from a point to the line. Squaring it stops overshoots and undershoots from cancelling.
Residuals for a slope that is too shallow. The further along x, the bigger the miss.
def predict(x, w, b):
    return w * x + b

xs = [1, 2, 3, 4]
ys = [2, 4, 6, 8]   # the true line is y = 2x
w = 1.5             # a too-shallow guess

for x, y in zip(xs, ys):
    r = predict(x, w, 0) - y
    print('x =', x, 'residual =', r, 'squared =', r * r)

Best fit means lowest error

Now you can state what a line of best fit actually is: among all the lines you could draw, it is the one whose total squared error is the smallest. That criterion is called least squares, and it is the definition every regression model inherits. The model never sees the right answer; it searches for the weights that minimise a number you can compute.

How does it search? One option is brute force — try candidate slopes, score each, and keep the best, as the final exercise does. That works for a single weight on a small grid, but it explodes once you have several features. The efficient route is to follow the slope of the error downhill — gradient descent — which the next two lessons build. Grasping that the goal is a minimum is what makes the method make sense.

A line versus a flat guess

Before you trust a line, ask whether it earns its slope. The simplest possible model ignores the feature entirely and always predicts the mean of the targets — a flat horizontal line, whose total squared error is just the spread (variance) of the data. A regression line is only worth keeping if it beats that flat guess; if the feature carries no real signal, the best slope lands near zero and the line collapses back toward the mean.

This is the seed of every goodness-of-fit measure: how much of the wrongness did using the feature actually remove? Squared error lets you answer that with one subtraction, and it is why mean squared error — not just error — became the default metric for regression. A model that cannot beat the mean has learned nothing.

flowchart TD
  T["try slope w = 0"] --> E0["score its<br/>total squared error"]
  T2["try slope w = 1"] --> E1["score it"]
  T3["try slope w = 2"] --> E2["score it"]
  E0 --> M["keep the slope<br/>with the lowest score"]
  E1 --> M
  E2 --> M
  style M fill:#166534,color:#fff
Least squares in one picture: score every candidate line, keep the one with the lowest error.
A brute-force search for the best slope. The lowest score wins — the seed of gradient descent.
def predict(x, w, b):
    return w * x + b

def total_error(xs, ys, w):
    return sum((predict(x, w, 0) - y) ** 2 for x, y in zip(xs, ys))

xs = [1, 2, 3]
ys = [2, 4, 6]
best_w = None
best_err = None
for w in [0, 1, 2, 3]:
    err = total_error(xs, ys, w)
    print('slope', w, '-> total squared error', err)
    if best_err is None or err < best_err:
        best_err = err
        best_w = w
print('best slope:', best_w)
Exercise

What does this print? The model applies a weight, then adds a bias.

def predict(x, w, b):
    return w * x + b

print(predict(3, 2, 1))
Exercise

Write predict(x, w, b) that returns the model's prediction for a single input using w * x + b.

def predict(x, w, b):
    # weight times feature, then add bias
    pass
Exercise

This prediction applies the bias to the feature BEFORE multiplying, which is wrong — the bias must be added after the weight has done its work. Fix the one line so the prediction follows w * x + b.

def predict(x, w, b):
    return w * (x + b)
Exercise

Write residual(x, y, w, b) returning the residual prediction - actual for one data point, where the prediction uses w * x + b and y is the actual target.

def residual(x, y, w, b):
    # prediction minus actual
    pass
Exercise

Write best_slope(xs, ys, candidates) that searches a list of candidate slopes (bias held at 0) and returns the one with the LOWEST total squared error. This is least squares by brute force — the idea gradient descent will soon do far faster.

def predict(x, w, b):
    return w * x + b

def total_error(xs, ys, w):
    # sum of (prediction - actual)^2 with bias 0
    pass

def best_slope(xs, ys, candidates):
    # return the candidate with the smallest total_error
    pass

Recap

  • Regression predicts a continuous number; the simplest model is a straight line prediction = w * x + b.
  • The weight sets the slope, the bias sets the baseline; the feature is the input you measure.
  • A residual is prediction - actual; summing raw residuals fails because overshoots and undershoots cancel.
  • Least squares defines the best line as the one with the lowest total squared error — the minimum that gradient descent will hunt for.
  • A line only earns its slope if it beats the flat mean prediction, and it only behaves inside the range you trained on.

Next you will turn that squared-error idea into a real metric — mean squared error and its root — so you can measure wrongness precisely before learning to minimise it.

Checkpoint quiz

A model predicts w * x + b. What does the bias b represent?

Why can you not judge a line by simply summing its residuals?

Go deeper — technical resources