Machine learning does not eat spreadsheets directly. It eats numbers arranged in a strict shape: a row for every example, a column for every measurable property, and a separate column for the answer you want to predict. That grid of inputs is the design matrix X; the list of answers is the target vector y. Turning a raw table into X and y is the first real engineering decision you make, and it shapes everything that follows. A model can only learn patterns that are visible in the features you choose. If the pattern is not in X, the most expensive algorithm in the world cannot find it.
Features, labels, and the design matrix
A feature is one measurable property of an example: the square footage of a house, the word count of an email, the hours studied before an exam. A label is the answer you want to predict: the sale price, whether the email is spam, the exam score.
If you have n examples and d features each, the design matrix X is a list of n rows, each row a list of d numbers. The target vector y is a list of n labels. Every row of X lines up with exactly one entry of y. Getting that alignment wrong — shuffling one list but not the other — is a bug that still compiles but produces nonsense.
examples = [
{"sqft": 1200, "bedrooms": 3, "price": 250000},
{"sqft": 1500, "bedrooms": 4, "price": 320000},
{"sqft": 900, "bedrooms": 2, "price": 180000},
]
X = [[e["sqft"], e["bedrooms"]] for e in examples]
y = [e["price"] for e in examples]
print("X =", X)
print("y =", y)
flowchart TD E1["Example 1:<br/>sqft=1200, beds=3"] --> R1["Row 1: [1200, 3]"] E2["Example 2:<br/>sqft=1500, beds=4"] --> R2["Row 2: [1500, 4]"] R1 --> Y1["Label 1: 250000"] R2 --> Y2["Label 2: 320000"] style R1 fill:#166534,color:#fff style R2 fill:#166534,color:#fff
Representation is a ceiling
A model cannot see what you do not measure. If you try to predict house prices using only the street name as a raw string, the model has no numerical signal to work with. If you measure square footage but not location, the model may learn that larger houses cost more on average, but it will miss the fact that a small apartment in a city centre can outsell a mansion in the countryside.
This is the representation problem: the features you choose are the only lens the model has. A bad lens makes a simple problem look impossible. A good lens — adding distance_to_city_centre or school_rating — can make a hard problem tractable. Feature engineering is the art of building that lens, and it is the single biggest lever on model performance most practitioners ever pull.
houses = [
{"sqft": 1200, "bedrooms": 3, "price": 250000},
{"sqft": 1500, "bedrooms": 4, "price": 320000},
]
# Representation A: raw sqft
X_a = [[h["sqft"]] for h in houses]
print("Representation A:", X_a)
# Representation B: sqft per bedroom (efficiency)
X_b = [[h["sqft"] / h["bedrooms"]] for h in houses]
print("Representation B:", [round(x[0], 2) for x in X_b])
flowchart LR R["Raw data"] --> F["Feature choices"] F --> D["Design matrix X"] D --> M["Model"] M --> P["Predictions"] style F fill:#b45309,color:#fff style D fill:#166534,color:#fff
Why rows must stay aligned
X and y are two separate lists, which means nothing in Python stops you from sorting one and forgetting the other. If you shuffle the houses but leave the prices in place, the model will learn that a three-bedroom house costs whatever the first price happens to be. That alignment is your responsibility.
In production, every row of X is processed independently, and the model has no idea what 'house number three' means. It only knows that the numbers in row three map to the label in y[3]. Guard that index like a password. A single misaligned pair corrupts the signal the model tries to learn, and the damage is invisible until you test on real data.
houses = [
{"sqft": 1200, "bedrooms": 3, "price": 250000},
{"sqft": 1500, "bedrooms": 4, "price": 320000},
]
X = [[h["sqft"], h["bedrooms"]] for h in houses]
y = [h["price"] for h in houses]
# Danger: sorting X but not y
X_sorted = sorted(X)
print("X_sorted:", X_sorted)
print("y (unchanged):", y)
What does this print? X[1] is the second row.
X = [[1, 2], [3, 4], [5, 6]] y = [10, 20, 30] print(X[1][0], y[1])
Python lists are zero-indexed.
X[1] is [3, 4], so X[1][0] is 3. y[1] is 20.
Write num_features(X) that returns the number of features (columns) in a design matrix X. Assume X has at least one row.
def num_features(X):
pass
Each row has the same number of entries.
The first row's length is the number of features.
def num_features(X):
return len(X[0])
Write build_matrix(examples, feature_keys) that takes a list of dict examples and a list of feature key names, and returns the design matrix X as a list of lists in the same order.
def build_matrix(examples, feature_keys):
pass
Loop over examples, then over feature_keys inside each example.
A nested list comprehension does this in one line.
def build_matrix(examples, feature_keys):
return [[e[k] for k in feature_keys] for e in examples]
This function builds X and y from a list of dicts, but it accidentally puts the label into X and leaves y empty. Fix it so X contains the feature values and y contains the label values.
def split_xy(examples, feature_keys, label_key):
X = [[e[label_key]] for e in examples]
y = []
return X, y
X should contain the feature keys, not the label key.
y should be a list of the label_key values.
def split_xy(examples, feature_keys, label_key):
X = [[e[k] for k in feature_keys] for e in examples]
y = [e[label_key] for e in examples]
return X, y
Write add_room_density(examples) that returns a NEW list of dicts where each dict has an extra key "sqft_per_room" equal to sqft / bedrooms. Do not modify the original dicts.
def add_room_density(examples):
pass
Use {**e, ...} to copy a dict and add a new key.
Divide sqft by bedrooms for each example.
def add_room_density(examples):
return [{**e, "sqft_per_room": e["sqft"] / e["bedrooms"]} for e in examples]
Recap
- A feature is a measurable input; a label is the target you want to predict.
- The design matrix
Xholds one row per example and one column per feature. - The target vector
yholds one label per row, aligned withX. - A model can only learn patterns that are visible in the features you choose; representation is a ceiling on performance.
- Never let the label leak into
X, and never include information you would not have at prediction time. Guard the alignment betweenXandyas carefully as you guard the code itself.