Logistic regression ends with a probability, but a real decision has to commit: is this email spam or not? At some point you stop hedging and pick a side. The decision boundary is the line in feature space where the model flips from favouring one class to favouring the other — the place it is genuinely torn.
That line is the most important thing to understand about your classifier. It shows where the model is confident, where it is only guessing, and — the part that catches people out — where its assumptions simply cannot fit the data. Here you will locate the boundary for a logistic model, see why it is always a straight line, and meet the shapes a line can never cut apart.
flowchart TD P["the feature plane<br/>plotted as x1 versus x2"] --> L["a straight line<br/>where the score is 0"] L --> Z["one side: class 0"] L --> O["other side: class 1"] style L fill:#10b981,color:#fff
Where the line comes from
The logistic model takes a score s = w0 + w1*x1 + w2*x2 and squashes it through the sigmoid into a probability. The rule for labelling is simple: predict class 1 when that probability reaches 0.5, and class 0 otherwise.
The decisive step is this: the sigmoid outputs exactly 0.5 when its input is 0. So the boundary is just the set of points whose score is zero — w0 + w1*x1 + w2*x2 = 0. Solve that equation for x2 and it becomes the familiar equation of a line: x2 = -(w0 + w1*x1) / w2. The whole story of a probability crossing a half collapses to a line you can draw with a ruler.
import math
def sigmoid(z):
return 1 / (1 + math.exp(-z))
w = [-2, 1, 1] # the score is w0 + w1*x1 + w2*x2
def boundary_x2(x1):
return -(w[0] + w[1] * x1) / w[2]
x1 = 1.0
print('at x1 =', x1, 'the boundary is x2 =', round(boundary_x2(x1), 2))
print('on the boundary the probability is', round(sigmoid(0), 2))
Confidence grows with distance
Most points do not sit exactly on the boundary. A point well inside class-1 territory has a large positive score, so the sigmoid drives its probability close to 1 — the model is sure. A point lying just across the line has a small score and a probability lingering near 0.5, where a nudge in either feature could flip the label.
That distance is called the margin, and it measures how much trust to place in a prediction. This is why counting right answers is not enough: a model can be correct yet uncertain, and those uncertain correct answers are the ones that break first when the data shifts slightly. Confident and right is what you actually want.
flowchart LR F["far from the line"] --> C1["very confident<br/>probability near 0 or 1"] N["near the line"] --> C2["uncertain<br/>probability near 0.5"] style C1 fill:#166534,color:#fff style C2 fill:#b45309,color:#fff
A straight line is all you get
Look at the boundary equation again: w0 + w1*x1 + w2*x2 = 0. Every feature appears only to the first power, multiplied by a weight and summed. That forces the boundary to be a straight line in two dimensions, and a flat plane in three. The model has no curve, no loop, no wiggle available to it.
This is the defining limit of a linear classifier: it separates only classes a single straight line can tell apart — data that is linearly separable. Tidy clusters on opposite sides of the plane? The line slides neatly between them. Classes woven into each other? No rotation of the line helps, and the model stays wrong on some points forever, however long it trains.
flowchart LR
subgraph S["linearly separable"]
S1["a line splits the two classes"] --> S2["accuracy can reach 1.0"]
end
subgraph X["XOR arrangement"]
X1["no single line works"] --> X2["best accuracy is 0.75"]
end
style S2 fill:#166534,color:#fff
style X2 fill:#b91c1c,color:#fff
def classify(w, x1, x2):
score = w[0] + w[1] * x1 + w[2] * x2
return 1 if score > 0 else 0
def accuracy(w, points, labels):
correct = 0
for (x1, x2), y in zip(points, labels):
if classify(w, x1, x2) == y:
correct += 1
return correct / len(points)
separable = [(-1, -1), (-2, -1), (1, 1), (2, 1)]
sep_labels = [0, 0, 1, 1]
print('separable accuracy:', accuracy([0, 1, 1], separable, sep_labels))
xor = [(0, 0), (1, 1), (0, 1), (1, 0)]
xor_labels = [0, 0, 1, 1]
print('XOR accuracy:', accuracy([0, 1, 1], xor, xor_labels))
What does this print? A point is class 1 when its score is greater than 0.
w = [-2, 3, 1] point = (1, 1) score = w[0] + w[1] * point[0] + w[2] * point[1] print(1 if score > 0 else 0)
First work out the score: w0 + w1x1 + w2x2.
A positive score means class 1; here the score is -2 + 3 + 1.
Write boundary_x2(w, x1) that returns the x2-coordinate of the decision boundary at a given x1, for a model with weights w = (w0, w1, w2). The boundary is the line w0 + w1*x1 + w2*x2 = 0, solved for x2.
def boundary_x2(w, x1):
# solve w0 + w1*x1 + w2*x2 = 0 for x2
pass
Rearrange w0 + w1x1 + w2x2 = 0 to x2 = -(w0 + w1*x1) divided by w2.
In Python, w[0] is w0, w[1] is w1, w[2] is w2.
def boundary_x2(w, x1):
return -(w[0] + w[1] * x1) / w[2]
This classify only puts a point in class 1 when its score exceeds 1, but the decision boundary is where the score is 0. Fix the threshold so a point is class 1 whenever it is on the positive side of the boundary.
def classify(w, point):
score = w[0] + w[1] * point[0] + w[2] * point[1]
return 1 if score > 1 else 0
The boundary is score == 0, so class 1 is the positive side: score greater than 0.
Change the 1 to a 0 in the comparison.
def classify(w, point):
score = w[0] + w[1] * point[0] + w[2] * point[1]
return 1 if score > 0 else 0
Write linear_accuracy(w, points, labels) that classifies each point with the linear rule (score w0 + w1*x1 + w2*x2; class 1 when the score is greater than 0, else 0) and returns the fraction classified correctly. The test on the XOR set expects 0.75, not 1.0 — proof a single line cannot separate it.
def linear_accuracy(w, points, labels):
# for each (x1, x2): score = w0 + w1*x1 + w2*x2, predict 1 if score > 0
# return the fraction of predictions that match the labels
pass
Loop over zip(points, labels), compute each score, and predict 1 when it is greater than 0.
Count the matches and divide by len(points).
def linear_accuracy(w, points, labels):
correct = 0
for (x1, x2), y in zip(points, labels):
score = w[0] + w[1] * x1 + w[2] * x2
pred = 1 if score > 0 else 0
if pred == y:
correct += 1
return correct / len(points)
Recap
- The decision boundary is where the classifier is undecided: for a logistic model, where the score is exactly 0 and the probability is 0.5.
- Because the score is linear in the features, the boundary is always a straight line (or a flat plane in more dimensions).
- Confidence tracks distance from the boundary: far points get probabilities near 0 or 1, close ones hover around 0.5.
- A linear classifier only separates linearly separable classes; woven patterns such as XOR cap out below perfect accuracy.
- On non-separable data the model does not error — it plateaus — so judge it with held-out accuracy, not just falling loss.
Next you will lift this two-class line into a world with many categories, voting between several boundaries at once.