Module 6 · Advanced Data Handling ⏱ 19 min

The collections Module

By the end of this lesson you will be able to:
  • Tally frequencies with Counter and read off the most common items
  • Group items by key with defaultdict, with no missing-key guard needed
  • Bundle fixed fields under readable names with namedtuple

Three jobs you keep rewriting

Some patterns turn up in almost every program. You count how often each word appears. You gather items into buckets keyed by some attribute. You bundle a few related fields and pass them around with readable names. Each is a few lines of fiddly code with a plain dict, and each hides a subtle bug, a missing key here, an unreadable positional index there.

Python's collections module ships ready-made tools for exactly these jobs. They are not shortcuts; they are the correct, tested implementations of patterns you would otherwise hand-roll and get wrong. Three of them earn a place in your daily toolkit: Counter, defaultdict, and namedtuple.

flowchart LR
  J["a counting job"] --> CT["Counter"]
  G["a grouping job"] --> DD["defaultdict"]
  B["a bundle of fields"] --> NT["namedtuple"]
  style CT fill:#3776ab,color:#fff
  style DD fill:#15803d,color:#fff
  style NT fill:#b45309,color:#fff
Each collections tool maps to one job: counting, grouping, or bundling fields.

Counter: counting, in one line

A Counter is a dict specialised for tallying. Feed it any iterable and it counts how many times each item appears, storing item to count. Counter("mississippi") hands you a mapping with 's': 4 and 'i': 4 ready to read.

Its superpower is most_common(n), which returns the n most frequent items as (item, count) pairs, with no sorting or key function to write. A missing key returns 0 rather than raising, so you can safely ask counts["rare"] without a guard. For anything that looks like a frequency or ranking problem, reach for Counter before you reach for a loop.

Counter tallies an iterable and ranks items with most_common().
from collections import Counter

inventory = Counter(["apple", "banana", "apple", "cherry", "apple"])
print(inventory)
print(inventory["apple"])
print(inventory.most_common(1))

Counters do arithmetic

Because a Counter is a mapping of counts, it supports set-like arithmetic directly. a + b adds counts for matching keys; a - b subtracts them, dropping any that fall to zero or below; a & b keeps the minimum count per key and a | b keeps the maximum. Adding Counter(a=2) to Counter(a=3) yields Counter(a=5) with no loop written by you.

This turns merging two frequency tables into a single expression. The subtraction form is especially handy for diffs: inventory - sold shows what remains, with exhausted items vanishing automatically. These operators only make sense for counts, which is precisely why they live on Counter rather than on a plain dict.

defaultdict: grouping without the guard

The grouping pattern with a plain dict is three steps every time: check whether the key exists, create an empty container if it does not, then append. Forget the check and the first append raises KeyError. A defaultdict removes that whole dance.

You build it with a factory, a callable that makes a fresh empty container, like list or int. When you access a missing key, the defaultdict calls the factory, stores the result under that key, and hands it back, all automatically. defaultdict(list) turns the grouping loop into two clean lines, and the key-check bug becomes impossible to write.

flowchart TD
  A["read groups of c"] --> B{"key exists?"}
  B -->|"yes"| C["return the value"]
  B -->|"no"| F["call factory list()"]
  F --> S["store empty list under c"]
  S --> C
  style F fill:#b45309,color:#fff
  style C fill:#15803d,color:#fff
defaultdict calls its factory to build an empty container for any missing key, then stores it.
defaultdict(list) groups words by first letter with no missing-key check.
from collections import defaultdict

groups = defaultdict(list)
for word in ["cat", "dog", "car"]:
    groups[word[0]].append(word)

print(dict(groups))
print(groups["z"])

namedtuple: fields with names

A tuple packs values, but you read them back by position, and record[2] tells no one what that value means. A namedtuple gives the same lightweight, immutable bundle readable field names: Point(x=3, y=4), read back as p.x and p.y.

It behaves like a tuple (indexable, unpackable, hashable) while also working like a tiny class with no boilerplate. Use it for fixed-shape records, a row of data, a coordinate, a config triple, where a full class would be overkill but bare tuples would hurt readability. To make a changed copy, call ._replace(x=10), which returns a new namedtuple with one field swapped.

A namedtuple reads back by name or by index, and _replace makes an altered copy.
from collections import namedtuple

Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)

print(p.x, p.y)
print(p[0])
print(p._replace(x=10))

Picking the right one

Each tool solves one job well, and using the wrong one is a common code smell. If you are counting, use Counter; a defaultdict(int) reinvents it without most_common. If you are collecting several values per key, use defaultdict(list). If you are grouping fixed fields under names, use namedtuple; if those fields need to change after creation, you wanted a dataclass, not a tuple.

The discipline pays off in readable code. A reader who sees Counter instantly knows the values are counts and that most_common is available; the type does the documenting, and the hand-rolled loop that hid a bug never gets written.

Exercise

Write counts(words) that returns a Counter of how many times each word appears. So counts(['a', 'b', 'a']) gives a Counter where 'a' maps to 2.

from collections import Counter

def counts(words):
    # build and return a Counter
    pass
Exercise

What does this print? Note how Counter treats a key that was never added.

from collections import Counter
c = Counter(["apple", "banana", "apple"])
print(c["apple"])
print(c["cherry"])
Exercise

Write group_by_length(words) that returns a plain dict mapping each length to the list of words of that length. Use defaultdict(list), then convert to dict at the end. For ["a", "bb", "ccc"] it returns {1: ["a"], 2: ["bb"], 3: ["ccc"]}.

from collections import defaultdict

def group_by_length(words):
    groups = defaultdict(list)
    # for each word: append it under groups[len(word)]
    return dict(groups)
Exercise

This groups words by first letter, but it crashes with KeyError the first time it sees a new letter, because a plain dict will not auto-create the list. Fix it by switching to defaultdict(list).

def group_by_first_letter(words):
    groups = {}
    for w in words:
        groups[w[0]].append(w)   # KeyError on the first sighting of each letter
    return groups
Exercise

Write grade_tally(students) that takes a list of student dicts like {'name': 'Ada', 'grade': 'A'} and returns a plain dict mapping each grade to how many students earned it. Use Counter on the grades, then convert to dict. So two 'A's and one 'B' gives {'A': 2, 'B': 1}.

from collections import Counter

def grade_tally(students):
    # count s['grade'] for each student s
    pass

Recap

  • Counter tallies an iterable and offers .most_common(n); missing keys read as 0, and counters add and subtract like sets.
  • defaultdict(factory) auto-creates an empty container for any missing key, turning the grouping pattern into a clean two-liner.
  • namedtuple gives immutable, name-accessible records with class-like readability and tuple-like simplicity.
  • Remember the side-effect contrast: defaultdict stores a default on read, while Counter and dict.get do not.

Next you will lock all of this down with tests, so you can prove your functions behave the way you claim.

Checkpoint quiz

You want the three most common words in a list. Which tool does it in one call?

After d = defaultdict(list), what is the side effect of reading d['new']?

Go deeper — technical resources