A model that memorises every example it sees will score perfectly on that data — and fail on everything else. This is the oldest trap in machine learning, and the only defence is a simple idea: hide some data from the model during training, then use that hidden data to ask how well the model really performs. That hidden data is the test set; the data the model learns from is the training set. Splitting correctly is not a detail — it is the difference between honest evaluation and self-deception. Every number you report to a colleague or a client depends on this split being clean.
The train/test split
The procedure is straightforward: shuffle the dataset so the order does not bias the split, then allocate a fraction — usually 70-80% — to training and the remainder to testing. The model never sees the test set during training, not even indirectly. If it did, the evaluation would be as dishonest as a student who peeked at the answer key before the exam.
The split must be reproducible: fixing the random seed guarantees that running the code again produces the exact same shuffle and the exact same split. Reproducibility is how you debug a model. If the score changes every time you run the script, you cannot tell whether a change you made helped or hurt, or whether you just got lucky with the shuffle.
import random
data = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
shuffled = data[:]
random.seed(42)
random.shuffle(shuffled)
split = int(len(data) * 0.8)
train = shuffled[:split]
test = shuffled[split:]
print("train:", train)
print("test:", test)
flowchart TD D["Full dataset"] --> S["Shuffle"] S --> T["Train set<br/>~80%"] S --> X["Test set<br/>~20%"] T --> M["Model trains<br/>on train only"] X --> E["Evaluate on<br/>unseen test"] style T fill:#166534,color:#fff style X fill:#b45309,color:#fff
Why shuffling matters
Real datasets are rarely in random order. Medical records may be sorted by admission date, sales data by region, and sensor logs by time. If you split without shuffling, the training set might contain only older patients and the test set only newer ones, or the training set only weekdays and the test set only weekends. The model would then fail not because it is bad, but because the two sets came from different distributions. Shuffling breaks up those hidden patterns and gives both splits a fair sample of the whole dataset.
Overfitting: when the model memorises instead of learning
A model with enough parameters can fit any training set perfectly by memorising every point, but that gives it no ability to generalise to new examples. The symptom is a large gap between training performance and test performance: the model looks brilliant on data it has seen and clueless on data it has not. That gap is called overfitting, and it is the reason every honest paper reports both numbers side by side.
The test set is your early-warning system. If training accuracy is 99% but test accuracy is 65%, the model has memorised noise rather than learning the underlying pattern. A smaller model, more data, or stronger regularisation — all topics for later lessons — are the usual cures. But you cannot diagnose any of them without a clean test set that the model has never touched, not even to choose between two candidate architectures.
# A 'model' that memorises perfectly on training data
train = [1, 2, 3, 4, 5]
test = [6, 7, 8]
def memorise(x, train_data):
return x if x in train_data else 0
train_score = sum(1 for x in train if memorise(x, train) == x) / len(train)
test_score = sum(1 for x in test if memorise(x, train) == x) / len(test)
print("train accuracy:", round(train_score, 2))
print("test accuracy:", round(test_score, 2))
flowchart TD O["Overfitting"] --> L["Train error: low"] O --> H["Test error: high"] L --> G["Gap = memorisation,<br/>not generalisation"] style O fill:#b91c1c,color:#fff style L fill:#166534,color:#fff style H fill:#b45309,color:#fff
Seeds and determinism
A random seed does not make the data more random; it makes the randomness repeatable. Setting random.seed(42) before a shuffle guarantees that every colleague who runs your script gets the exact same train and test sets. That repeatability is essential for debugging: if you change the model and the score changes, you want to know the change came from the model, not from a different shuffle. In research and production, reproducibility is not optional — it is the baseline that makes science possible. Never report a score you cannot reproduce; if you cannot reproduce it, you do not understand it.
What does this print? An 80% split of ten items.
data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] n_train = int(len(data) * 0.8) print(n_train)
80% of 10 is 8.
int(8.0) is 8.
Write train_test_split(data, test_size, seed) that returns (train, test). It should shuffle a COPY of data using random.seed(seed) and random.shuffle, then put the first len(data) - test_size elements into train and the rest into test. test_size is an integer number of examples.
import random
def train_test_split(data, test_size, seed):
pass
Copy the list with data[:] before shuffling.
Set the seed before shuffle so the result is reproducible.
import random
def train_test_split(data, test_size, seed):
shuffled = data[:]
random.seed(seed)
random.shuffle(shuffled)
split = len(data) - test_size
return shuffled[:split], shuffled[split:]
Write split_by_ratio(data, train_ratio, seed) that returns (train, test) where train_ratio is a float between 0 and 1 (e.g. 0.8). Compute the split point with int(len(data) * train_ratio).
import random
def split_by_ratio(data, train_ratio, seed):
pass
Shuffle a copy first, then compute the split index.
Return the slices before and after the split index.
import random
def split_by_ratio(data, train_ratio, seed):
shuffled = data[:]
random.seed(seed)
random.shuffle(shuffled)
split = int(len(data) * train_ratio)
return shuffled[:split], shuffled[split:]
This function splits data into train and test, but it forgets to shuffle first. If the data is sorted, the test set will contain only the largest values. Fix it by shuffling a copy of the data before splitting.
import random
def bad_split(data, test_size, seed):
random.seed(seed)
split = len(data) - test_size
return data[:split], data[split:]
Copy the list before shuffling.
Use random.shuffle on the copy, not the original.
import random
def bad_split(data, test_size, seed):
shuffled = data[:]
random.seed(seed)
random.shuffle(shuffled)
split = len(data) - test_size
return shuffled[:split], shuffled[split:]
Write train_test_split_indices(n, test_size, seed) that returns two lists: the indices for the train set and the indices for the test set, in the order they appear after shuffling. Use list(range(n)) as the starting list of indices.
import random
def train_test_split_indices(n, test_size, seed):
pass
Create indices with list(range(n)), then shuffle them.
Split at n - test_size.
import random
def train_test_split_indices(n, test_size, seed):
indices = list(range(n))
random.seed(seed)
random.shuffle(indices)
split = n - test_size
return indices[:split], indices[split:]
Recap
- The training set is what the model learns from; the test set is what it is judged on.
- Shuffle first, then split, to avoid bias from sorted or grouped data. Real-world ordering is never random.
- Fix the random seed so the split is reproducible and debuggable. If you cannot reproduce a result, you do not understand it.
- A large gap between training and test performance signals overfitting: the model memorised instead of generalising.
- The test set must be untouched during model development; tuning on it makes the evaluation worthless and the model unreliable in production.