Module 3 · Regression from Scratch ⏱ 18 min

Gradient Descent, Built by Hand

By the end of this lesson you will be able to:
  • Explain gradient descent as repeatedly stepping downhill on the loss surface
  • Implement the mean-squared-error loss and its gradient for linear regression
  • Write the update rule that adjusts a weight in the direction that lowers loss
  • Run the loop and watch the loss fall toward a fit

Almost every model in machine learning is trained by the same idea, and it is simpler than it sounds: start with a bad guess, measure how wrong it is, then nudge the guess in the direction that makes it a little less wrong — and repeat. That direction is given by the gradient, the slope of the error, and following it downhill is gradient descent.

It is the engine under linear regression, logistic regression, and every neural network. Libraries hide it behind .fit(), but the whole thing is a short loop you can write yourself. In this lesson you build it for the simplest case — fitting a single weight — so that when you later call an optimiser, you know exactly what it is doing and why it sometimes fails.

The loss: how wrong are we?

Before you can go downhill you need to know which way is down. That is what a loss function gives you: a single number for how badly the current model fits the data. For predicting numbers, the standard choice is mean squared error (MSE) — the average of the squared differences between predictions and true values.

Squaring does two jobs: it makes every error positive (so they cannot cancel out), and it punishes big misses far more than small ones. A model that is off by 4 on one point contributes 16, while four points off by 1 each contribute only 4 in total. Minimising MSE therefore pulls hardest against the worst mistakes.

Mean squared error: average of the squared gaps. Press Run.
def mse(predictions, targets):
    total = 0.0
    for p, t in zip(predictions, targets):
        total += (p - t) ** 2
    return total / len(predictions)

print(round(mse([2, 4, 6], [2, 4, 6]), 2))   # perfect fit
print(round(mse([2, 4, 6], [3, 4, 5]), 2))   # off by 1 on two points
flowchart LR
  A["guess a weight"] --> B["measure loss"]
  B --> C["compute gradient<br/>(slope of loss)"]
  C --> D["step opposite the slope"]
  D --> B
  B --> E["loss stops falling:<br/>stop"]
  style C fill:#10b981,color:#fff
  style E fill:#166534,color:#fff
Gradient descent: measure the slope of the loss, step the opposite way, repeat until the loss stops falling.

The gradient: which way is downhill?

Model a line through the origin, prediction = w * x, with one weight w. The loss is a function of w: pick a w, get a number. If you plotted loss against w you would see a bowl, and training means rolling to the bottom of that bowl.

The gradient is the slope of that bowl at your current w — it tells you which way the loss rises. For MSE the slope works out to (2/n) * sum((w*x - y) * x) over the data. You do not need to derive it here; you need to know its sign points uphill, so to go down you step in the opposite direction.

One gradient step lowers the loss. Run it and watch the number fall.
xs = [1, 2, 3, 4]
ys = [2, 4, 6, 8]   # the true relationship is y = 2x

def loss(w):
    return sum((w * x - y) ** 2 for x, y in zip(xs, ys)) / len(xs)

def gradient(w):
    return (2 / len(xs)) * sum((w * x - y) * x for x, y in zip(xs, ys))

w = 0.0
lr = 0.01
print('start  w =', round(w, 3), 'loss =', round(loss(w), 3))
w = w - lr * gradient(w)
print('step 1 w =', round(w, 3), 'loss =', round(loss(w), 3))

The update rule

The single line that trains the model is the update rule: w = w - learning_rate * gradient. The gradient sets the direction; the learning rate sets how big a step you take. Subtracting moves you against the uphill slope — that is, downhill.

Run that line over and over and w walks toward the value that minimises the loss. Here the data was y = 2x, so a healthy loop should drive w toward 2 and the loss toward 0. Watching those two numbers move is how you tell training is working — and, later, how you diagnose it when it is not.

The full loop: 200 steps march w toward 2 and the loss toward 0. Run it.
xs = [1, 2, 3, 4]
ys = [2, 4, 6, 8]

def loss(w):
    return sum((w * x - y) ** 2 for x, y in zip(xs, ys)) / len(xs)

def gradient(w):
    return (2 / len(xs)) * sum((w * x - y) * x for x, y in zip(xs, ys))

w = 0.0
lr = 0.01
for step in range(1, 201):
    w = w - lr * gradient(w)
    if step % 50 == 0:
        print('step', step, 'w =', round(w, 3), 'loss =', round(loss(w), 4))
flowchart TD
  S["learning rate"] --> A["too small:<br/>loss falls very slowly"]
  S --> B["just right:<br/>steady descent to the min"]
  S --> C["too big:<br/>overshoots, loss can grow"]
  style A fill:#b45309,color:#fff
  style B fill:#166534,color:#fff
  style C fill:#b91c1c,color:#fff
The learning rate decides the step size: too small crawls, too big overshoots and can diverge.
Exercise

What does this print? MSE is the average of the squared differences.

def mse(predictions, targets):
    total = 0.0
    for p, t in zip(predictions, targets):
        total += (p - t) ** 2
    return total / len(predictions)

print(round(mse([1, 2], [3, 2]), 2))
Exercise

Write mse(predictions, targets) returning the mean of the squared differences between the two equal-length lists.

def mse(predictions, targets):
    # average of (p - t) squared
    pass
Exercise

Write predict(xs, w) that returns a list of predictions for the single-weight model prediction = w * x, one per value in xs.

def predict(xs, w):
    # return [w * x for each x]
    pass
Exercise

This update step ADDS the learning-rate times the gradient, so it walks UPHILL and the loss grows. Gradient descent must step AGAINST the slope. Fix the one operator so the weight moves toward a lower loss.

def step(w, grad, lr):
    return w + lr * grad
Exercise

Write gradient(xs, ys, w) for the single-weight model, returning (2/n) * sum((w*x - y) * x) over the data. Then train(xs, ys, w, lr, steps) should apply the update rule w = w - lr * gradient(...) that many times and return the final w. xs and ys are equal-length lists.

def gradient(xs, ys, w):
    # (2/n) * sum((w*x - y) * x)
    pass

def train(xs, ys, w, lr, steps):
    # apply w = w - lr * gradient(xs, ys, w), `steps` times
    pass

Recap

  • Gradient descent trains a model by repeatedly measuring the loss and stepping in the direction that lowers it.
  • MSE — the mean of squared errors — is the loss for predicting numbers; squaring punishes big misses hardest.
  • The gradient is the slope of the loss with respect to the weight; it points uphill, so you step the opposite way.
  • The update rule is one line: w = w - learning_rate * gradient. The gradient gives direction, the learning rate gives step size.
  • A too-large learning rate makes the loss diverge (nan/infinity); a too-small one makes it crawl. Tuning it is a permanent part of the job.

You just wrote the core loop of nearly all of machine learning. Next you will extend it from one weight to many, and then to classification with logistic regression — the same engine, a different loss.

Checkpoint quiz

In the update rule w = w - learning_rate * gradient, why is the gradient SUBTRACTED?

Your training loss quickly grows to nan. What is the most likely cause?

Why does MSE square the errors instead of just averaging them?

Go deeper — technical resources