The last lesson ended with a problem: a raw straight line can output -0.1 or 1.3, and neither is a legal probability. We need a function that takes any real number the line produces and gently squashes it into the range between 0 and 1, treating very negative inputs as near-certain no and very positive inputs as near-certain yes. That function exists, it is famous, and it is short. It is called the sigmoid, and once you have it, classification stops being a hack and becomes honest probability estimation. Every logistic regression and every neuron in a neural network leans on a cousin of this exact curve, so the time you spend here pays off for the rest of the course.
The sigmoid, in one line
The sigmoid (also called the logistic function) of a number z is 1 / (1 + exp(-z)). That is the whole definition. Read it slowly: take the input z, flip its sign to get -z, raise e to that power, add one, and divide one by the result. The exp is just e raised to a power, and Python hands it to you as math.exp. Two symbols of math and one library call are the entire mechanism — everything else in this lesson is about why this particular shape is exactly what a classifier wants.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
for z in [-10, -1, 0, 1, 10]:
print('sigmoid(' + str(z) + ') =', round(sigmoid(z), 4))
Why it stays between 0 and 1
The denominator 1 + exp(-z) is what does the trapping. exp(-z) is always positive, so the denominator is always greater than 1, which means the fraction is always less than 1. And because the denominator is always at least 1, the fraction is always greater than 0. So the output is locked strictly between 0 and 1 for every input you can name — precisely the property a probability must have. Push z toward large positive values and exp(-z) shrinks toward 0, so the output coasts up toward 1. Push z negative and exp(-z) balloons, dragging the output down toward 0. These are asymptotes, not endpoints: the curve approaches 0 and 1 forever but never quite lands, which is right for a probability that should never claim total certainty.
flowchart LR A["z very negative<br/>sigma near 0"] --> B["z = 0<br/>sigma = 0.5"] B --> C["z very positive<br/>sigma near 1"] style A fill:#b91c1c,color:#fff style B fill:#b45309,color:#fff style C fill:#166534,color:#fff
Symmetric about a coin flip
Plug in z = 0 and you get 1 / (1 + exp(0)), which is 1 / 2, or 0.5 — exactly a coin flip. That is no accident: the sigmoid is symmetric in the sense that sigma(-z) equals 1 - sigma(z). Knowing the value at +2 tells you the value at -2 for free. The point z = 0 is special in another way too: it is where the model is perfectly undecided, so it is the natural decision boundary. Above z = 0 the probability of the positive class exceeds half; below it, the negative class wins. One number, 0.5, splits the whole line.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
for z in [0.5, 2.0, 5.0]:
a = sigmoid(z)
b = 1 - sigmoid(-z)
print('z =', z, 'sigma =', round(a, 4), '1 - sigma(-z) =', round(b, 4))
From a score to a probability
The sigmoid is the second half of a two-step recipe. Step one is the linear score z = w times x plus b, exactly the line you already know — it can be any real number. Step two is to feed that score through the sigmoid, which squeezes it into a probability. A big positive score becomes a probability near 1; a big negative score becomes a probability near 0; a score of zero becomes 0.5. The line supplies a confidence in a direction; the sigmoid translates that confidence into a number you can read as a probability, threshold at 0.5, and turn into a decision.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
def score(hours, w, b):
return w * hours + b
w, b = 0.5, -2.5
for h in [2, 5, 8]:
p = sigmoid(score(h, w, b))
print('hours =', h, '-> p(pass) =', round(p, 4))
flowchart TD Z["input z = w*x + b<br/>(any real number)"] --> E["compute exp(-z)"] E --> D["denominator 1 + exp(-z)"] D --> S["sigma = 1 / denominator<br/>always between 0 and 1"] S --> P["read as P(positive class)"] style S fill:#10b981,color:#fff style P fill:#166534,color:#fff
Saturation: confidence has a cost
Notice that the sigmoid is steepest near z = 0 and flattens out toward both ends. Far from zero, huge changes in z barely move the probability — it has saturated, pinned near 0 or 1. Saturation is why a well-trained classifier can become highly confident, and it is also the seed of a famous problem in deep networks called the vanishing gradient: when a neuron is saturated, learning slows to a crawl because the sigmoid's slope is nearly flat there, so the update signal fades. For now the takeaway is simpler: once the model is sure, extra evidence changes its mind very little.
flowchart TD Z0["z = 0<br/>sigma = 0.5"] --> D["decision boundary:<br/>exactly undecided"] ZP["z large positive<br/>sigma near 1"] --> SAT1["saturated: confident<br/>in the positive class"] ZN["z large negative<br/>sigma near 0"] --> SAT0["saturated: confident<br/>in the negative class"] style D fill:#b45309,color:#fff style SAT1 fill:#166534,color:#fff style SAT0 fill:#b91c1c,color:#fff
What does this print? Each line is rounded to four decimals. Recall sigma(0) = 0.5.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
print(round(sigmoid(0), 4))
print(round(sigmoid(2), 4))
print(round(sigmoid(-2), 4))
sigma(0) = 1 / (1 + 1) = 0.5, which prints as 0.5.
sigma(2) = 1 / (1 + exp(-2)); exp(-2) is about 0.135, so this is about 0.8808.
By symmetry sigma(-2) = 1 - sigma(2).
Write sigmoid(z) using math.exp. It must return 1 / (1 + math.exp(-z)). Remember to import math.
import math
def sigmoid(z):
# 1 divided by (1 + exp(-z))
pass
Import math at the top, then return the formula.
math.exp(0) is exactly 1.0, so sigmoid(0) is exactly 0.5.
Far from zero the output stays inside (0, 1) but never reaches the endpoints.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
Write probability(x, w, b) that first computes the linear score z = w * x + b, then returns sigmoid(z). Include both sigmoid and the import so your code is self-contained.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
def probability(x, w, b):
# score z = w * x + b, then sigmoid
pass
Build z from the parameters, then hand z to sigmoid.
With w = 1 and b = 0, the score equals x itself.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
def probability(x, w, b):
return sigmoid(w * x + b)
This sigmoid has the wrong sign in the exponent: it computes exp(z) instead of exp(-z), so it returns the complement of the true probability and gets positive and negative backwards. Fix the one argument so a positive z yields a probability above 0.5.
import math
def sigmoid(z):
return 1 / (1 + math.exp(z))
For a large positive z you expect a probability near 1, not near 0.
The exponent must be -z so that positive z drives exp toward 0.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
A linear score z feeds the sigmoid. What probability does the model output when z = 0?
sigma(0) = 1 / (1 + exp(0)) = 1 / 2 = 0.5. The input z = 0 is the decision boundary: the sigmoid is balanced between the classes, so a classifier thresholds it right at the coin-flip point.
Recap
- The sigmoid
1 / (1 + exp(-z))squashes any real number into the open range (0, 1), making it a valid probability. - It equals 0.5 at z = 0 — the natural decision boundary where the model is undecided.
- It is symmetric: sigma(-z) = 1 - sigma(z), so the curve mirrors around the 0.5 line.
- A linear score z = w * x + b feeds the sigmoid to turn a direction-and-magnitude into a probability.
- It saturates far from zero: once confident, extra input barely moves the probability.
- The naive form overflows for very negative z; the stable two-branch form avoids the crash.
Next you will bolt this sigmoid onto gradient descent to build logistic regression — a classifier that learns its own weights by minimising a loss designed for probabilities.