Module 6 · Advanced Data Handling ⏱ 18 min

Deeply Nested Data Structures

By the end of this lesson you will be able to:
  • Read values out of a deeply nested dict-and-list structure using chained indexing
  • Explain why a chained access can raise KeyError or TypeError, and tell them apart
  • Navigate API-like data safely with .get() defaults, try/except, and a helper

Real data is shaped like a tree, not a table. A weather service returns a dict of cities; each city is a dict of readings; each reading holds a list of hourly values. A payment record nests a customer inside an order inside a transaction. One missing branch anywhere in that tree and the single line of code that reads it raises an exception, halting your whole pipeline midway through.

The skill that separates robust code from fragile code is safe navigation: reaching into a deeply nested structure without crashing when a branch is absent or the wrong type. This lesson teaches the mechanics of chained access and, more importantly, the idioms that keep a program running when the data is imperfect, because outside the textbook it always is.

flowchart TD
  R["API response"] --> U["user"]
  U --> N["name: 'Ada'"]
  U --> A["address"]
  A --> C["city: 'London'"]
  A --> G["geo"]
  G --> LAT["lat: 51.5"]
  style R fill:#3776ab,color:#fff
  style A fill:#1e293b,color:#fff
An API response is a tree: dicts and lists nested several layers deep.

Reaching in, one bracket at a time

You read a nested structure with chained indexing — square brackets in a row, evaluated left to right. Each bracket peels off one layer and returns whatever lived there, and the type of that result decides what the next bracket is allowed to do.

response = {"user": {"address": {"city": "London"}}}
print(response["user"]["address"]["city"])   # London

Read it outward in: response["user"] yields the inner dict, that dict's ["address"] yields the next, and the final ["city"] yields the string. Count your brackets against the depth of the data and you will rarely go wrong, until a branch is missing, which is the next idea.

Chain brackets to walk down through the layers of a response.
response = {
    "user": {
        "name": "Ada",
        "address": {"city": "London", "geo": {"lat": 51.5}}
    }
}

print(response["user"]["name"])
print(response["user"]["address"]["city"])
print(response["user"]["address"]["geo"]["lat"])

Look at the shape before you dig

A habit worth forming the moment you meet unfamiliar data: parse it, then print the whole object and study its shape before writing the line that indexes into it. You cannot chain brackets confidently through a structure whose layers you have never seen. Once you know which keys exist at each level and which hold lists versus dicts, the chained read almost writes itself.

This is also how you read other people's code that already navigates the data: trace one bracket at a time, asking what type each step returns. Misreading the shape of one layer is the root cause of most nested-data crashes.

The two ways it breaks

A chained read fails in one of two ways, and telling them apart is half of defensive coding. The first is the KeyError: you asked for a key the dict does not have, so address["city"] on a record with no city raises immediately. The second is sneakier, the TypeError, which happens when a bracket lands on something that is not subscriptable at all, most often None.

data = {"user": None}
data["user"]["name"]   # TypeError: NoneType is not subscriptable

A server with no user data often sends null, Python's None, rather than omitting the key entirely, so user exists but cannot be indexed. Both exceptions stop the program dead; the rest of this lesson is about catching them.

flowchart TD
  Q["data user city"] --> T1{"'user' present?"}
  T1 -->|"no"| KE["KeyError"]
  T1 -->|"yes"| T2{"value a dict?"}
  T2 -->|"no, it is None"| TE["TypeError"]
  T2 -->|"yes"| OK["the city value"]
  style KE fill:#b91c1c,color:#fff
  style TE fill:#b45309,color:#fff
  style OK fill:#15803d,color:#fff
A chained read fails two ways: KeyError on a missing key, TypeError when a layer is None.
dict.get returns None for a missing key instead of crashing.
record = {"name": "Ada", "email": "ada@example.com"}

print(record.get("name"))
print(record.get("email"))
print(record.get("phone"))
print(record.get("phone", "no phone on file"))

.get() trades a crash for a None

The simplest guard is the dict method .get(key). It returns the value when the key exists and None when it does not, never a KeyError. Hand it a second argument and that becomes the default instead of None, so record.get("phone", "unknown") returns a friendly string for a missing field.

This is perfect for the top layer of a lookup, where a missing key is a normal event. It does not, on its own, solve deep chains: if an intermediate value is None, calling .get() on that None raises an AttributeError, the third failure mode hiding one layer down. For shallow reads, though, .get() is the right tool and the one you will reach for most often.

A helper that returns the value when the path exists and None when anything along it is missing.
def safe_city(response):
    try:
        return response["user"]["address"]["city"]
    except (KeyError, TypeError):
        return None

print(safe_city({"user": {"address": {"city": "London"}}}))
print(safe_city({"user": {}}))
print(safe_city({}))
print(safe_city({"user": None}))

Wrap the whole chain, not each step

The safe_city helper above is the Pythonic shape for deep reads: attempt the full chain in one go inside a try, and catch the two exceptions a missing or mistyped layer can raise. A KeyError means a dict lacked a key; a TypeError means an intermediate was None or a list; either way the data is not there, so returning a default is the honest answer.

This reads far better than a tower of if checks, and it scales to any depth for the price of one try. Pull the chain into a named function and your calling code stops worrying about shape entirely; it just asks for the city and gets either a string or None. That boundary, messy data in and clean values out, is exactly what robust code looks like.

Exercise

Write get_lat(response) that returns the latitude stored deep inside: response['geo']['lat']. Assume the keys are always present for this drill.

def get_lat(response):
    # chain two brackets: response['geo']['lat']
    pass
Exercise

What does this print? Two brackets reach a list element nested under a key.

data = {"user": {"name": "Ada", "scores": [10, 20, 30]}}
print(data["user"]["name"])
print(data["user"]["scores"][1])
Exercise

Write first_tag(record) that returns the first item from the record's tags list. If tags is missing or empty, return the string "none". Use .get() so a missing key never crashes.

def first_tag(record):
    # fetch tags with record.get('tags'); it is None if absent
    pass
Exercise

This crashes with KeyError whenever a record has no contact. Fix it so a missing contact, or a contact with no phone, returns None instead of raising. (Hint: use .get() at each level.)

def get_phone(record):
    return record["contact"]["phone"]   # crashes if 'contact' is absent
Exercise

Write collect_cities(records) that takes a list of record dicts and returns a list of every city that is present. Each record may have an address dict holding a city, or it may have neither, or address may be empty. Skip any record whose city is missing; do not crash on any shape.

def collect_cities(records):
    result = []
    # for each record: safely read address, then city, then append if present
    return result

Recap

  • Nested data is a tree of dicts and lists; read it with chained indexing, one bracket per layer, left to right.
  • A chained read fails two main ways: KeyError (a missing key) and TypeError (indexing None or a non-subscriptable value).
  • .get(key, default) turns a missing top-level key into a safe None or default instead of a crash.
  • For deep chains, prefer a helper built on try/except (KeyError, TypeError) or an isinstance check; never assume an intermediate value is a dict.

Next you will turn data into a stream with generators, so even huge nested structures can be processed without loading them entirely into memory.

Checkpoint quiz

What does data['user']['name'] raise if data is {"user": None}?

Why does data.get('user').get('city') still risk a crash on deep data?

Go deeper — technical resources