Almost every useful prediction depends on more than one number. A house price is not just square footage; it is bedrooms, bathrooms, age, and location combined. The single-weight model you built last lesson is the right intuition, but real regression uses many features at once, each weighted by how much it pushes the prediction up or down.
This is multiple linear regression — the same gradient-descent engine, now with a whole vector of weights to tune and a bias term that sets the baseline. The mechanics are barely more complex than the one-weight case: compute the error, find the gradient for each weight, and step downhill. But the power multiplies, because the model can now learn from every column in your dataset at once.
flowchart LR F1["sqft"] --> W1["weight 1"] F2["bedrooms"] --> W2["weight 2"] F3["age"] --> W3["weight 3"] W1 --> S["sum + bias"] W2 --> S W3 --> S S --> P["prediction"] style S fill:#10b981,color:#fff
The prediction formula
With n features, the model is prediction = bias + w1*x1 + w2*x2 + ... + wn*xn. Each wi scales its matching feature xi, and the bias is the prediction when every feature is zero. In Python you compute this with a loop or with zip to pair weights with features.
The weights are not directly comparable unless the features share the same scale. A weight of 50 on bedrooms sounds enormous next to a weight of 0.1 on square footage, but if bedrooms are counted in ones and square footage in thousands, the actual dollar contribution can be identical. The meaning of a weight is always weight * typical_feature_value, never the raw number alone.
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
house = [2000, 3, 10] # sqft, bedrooms, age
weights = [0.1, 10.0, -5.0]
bias = 50.0
print(round(predict(house, weights, bias), 2))
Run the code and you see a concrete prediction. The square footage contributes 2000 * 0.1 = 200, the three bedrooms contribute 30, and the ten years of age subtract 50. The bias adds a floor of 50. Every feature pulls the final number in its own direction, and the weights control the strength of that pull. Changing the weight on age from -5 to -1 would make the model treat old houses far more leniently, even though the data has not changed.
flowchart LR W1["weight 50\nbedrooms"] --> C1["contribution = 50 * 2 = 100"] W2["weight 0.1\nsqft"] --> C2["contribution = 0.1 * 2000 = 200"] C1 --> R["sqft matters more,\ndespite smaller weight"] style R fill:#b45309,color:#fff
Updating every weight at once
The loss is still mean squared error, but the error now depends on every weight and the bias. The gradient with respect to weight wi is (2/n) * sum(error * xi) where xi is that feature's value across all examples. The bias gradient is the same with xi replaced by 1, because the bias acts like a weight on a phantom feature that is always 1.
You loop over the weights, compute each gradient from the shared errors, and apply the update rule. The learning rate is shared across every weight — one scalar steps them all — which is exactly why feature scaling matters. An unscaled feature with huge values produces huge gradients, so its weight oscillates while others barely move. A well-scaled dataset lets every weight descend together.
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
def gradient_step(features_list, targets, weights, bias, lr):
n = len(targets)
errors = [predict(f, weights, bias) - t for f, t in zip(features_list, targets)]
bias_grad = (2 / n) * sum(errors)
weight_grads = [(2 / n) * sum(errors[i] * features_list[i][j] for i in range(n)) for j in range(len(weights))]
new_bias = bias - lr * bias_grad
new_weights = [w - lr * g for w, g in zip(weights, weight_grads)]
return new_weights, new_bias
data = [[1, 2], [2, 3], [3, 4]]
targets = [5, 8, 11]
weights = [0.0, 0.0]
bias = 0.0
w, b = gradient_step(data, targets, weights, bias, 0.01)
print('weights:', [round(x, 3) for x in w])
print('bias:', round(b, 3))
flowchart TD D["dataset with\nm features"] --> G["compute gradient\nfor each weight"] G --> U1["bias = bias - lr * grad_bias"] G --> U2["w1 = w1 - lr * grad_w1"] G --> U3["w2 = w2 - lr * grad_w2"] U1 --> L["loss lower?\nrepeat"] U2 --> L U3 --> L style G fill:#10b981,color:#fff
Reading the weights
After training, a positive weight means that increasing the feature increases the prediction; a negative weight means the opposite. A weight near zero means the feature did not help much, or it was redundant with another feature. But remember: if one feature is measured in thousands and another in fractions, the numeric size of the weight is misleading. To compare importance fairly you must either scale the features first, or multiply each weight by the feature's standard deviation. The model learns the numbers that minimise loss; it does not care about human-readable magnitudes.
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
def train_step(features_list, targets, weights, bias, lr):
n = len(targets)
errors = [predict(f, weights, bias) - t for f, t in zip(features_list, targets)]
bias_grad = (2 / n) * sum(errors)
weight_grads = [(2 / n) * sum(errors[i] * features_list[i][j] for i in range(n)) for j in range(len(weights))]
new_bias = bias - lr * bias_grad
new_weights = [w - lr * g for w, g in zip(weights, weight_grads)]
return new_weights, new_bias
data = [[1, 2], [2, 1], [3, 3], [4, 2]]
targets = [5, 4, 9, 8] # exactly y = 1*x1 + 2*x2
weights = [0.0, 0.0]
bias = 0.0
lr = 0.01
for step in range(1, 4):
weights, bias = train_step(data, targets, weights, bias, lr)
print('step', step, 'weights =', [round(w, 3) for w in weights], 'bias =', round(bias, 3))
What does this print? A two-feature prediction with bias.
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
print(round(predict([2, 3], [1.0, 2.0], 5.0), 2))
5 + (1.0 * 2) + (2.0 * 3) = 5 + 2 + 6.
The bias is added after the weighted sum.
Write predict(features, weights, bias) that returns the prediction for a single example. features and weights are equal-length lists.
def predict(features, weights, bias):
# bias + sum of w * x
pass
Use zip to pair each weight with its feature.
Add the bias after the sum.
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
Write mse_multi(features_list, targets, weights, bias) that returns the mean squared error for a multi-feature model. features_list is a list of feature lists; targets is a list of numbers.
def mse_multi(features_list, targets, weights, bias):
# average of (prediction - target)^2
pass
Compute each prediction with the weighted-sum formula.
Average the squared differences.
def mse_multi(features_list, targets, weights, bias):
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
total = sum((predict(f, weights, bias) - t) ** 2 for f, t in zip(features_list, targets))
return total / len(targets)
This gradient function forgets the bias term when computing predictions, so the gradients are wrong. Add bias to the prediction inside the error computation.
def gradients(features_list, targets, weights, bias):
n = len(targets)
errors = [sum(w * x for w, x in zip(weights, f)) - t for f, t in zip(features_list, targets)]
bias_grad = (2 / n) * sum(errors)
weight_grads = [(2 / n) * sum(errors[i] * features_list[i][j] for i in range(n)) for j in range(len(weights))]
return weight_grads, bias_grad
The prediction must include bias + sum(w * x).
Only the first line is wrong.
def gradients(features_list, targets, weights, bias):
n = len(targets)
errors = [bias + sum(w * x for w, x in zip(weights, f)) - t for f, t in zip(features_list, targets)]
bias_grad = (2 / n) * sum(errors)
weight_grads = [(2 / n) * sum(errors[i] * features_list[i][j] for i in range(n)) for j in range(len(weights))]
return weight_grads, bias_grad
Write train(features_list, targets, weights, bias, lr, steps) that runs gradient descent for the given number of steps and returns (final_weights, final_bias) as a tuple. Include a helper predict inside your solution so it is self-contained.
def train(features_list, targets, weights, bias, lr, steps):
# return (weights, bias) after gradient descent
pass
Compute errors as prediction minus target for every example.
Update bias and weights every step before moving to the next.
def train(features_list, targets, weights, bias, lr, steps):
def predict(features, weights, bias):
return bias + sum(w * x for w, x in zip(weights, features))
n = len(targets)
for _ in range(steps):
errors = [predict(f, weights, bias) - t for f, t in zip(features_list, targets)]
bias_grad = (2 / n) * sum(errors)
weight_grads = [(2 / n) * sum(errors[i] * features_list[i][j] for i in range(n)) for j in range(len(weights))]
bias = bias - lr * bias_grad
weights = [w - lr * g for w, g in zip(weights, weight_grads)]
return weights, bias
Recap
- Multiple regression combines many features in a weighted sum:
prediction = bias + sum(w_i * x_i). - The gradient for each weight is
(2/n) * sum(error * x_i); the bias gradient is(2/n) * sum(error). - Gradient descent updates every weight simultaneously with the same learning rate, so feature scaling keeps descent balanced.
- A weight's sign tells you the direction of influence, but its magnitude only matters relative to the feature's scale.
- Next you will see what happens when the true relationship is not a straight line at all — and how new features can capture curves, or overfit them.