PYTHON / ITERATORS, GENERATORS, AND COMPREHENSIONS
yield from and delegation
Use yield from to delegate iteration, send/throw/close and a subgenerator's return value instead of hand-writing a forwarding loop.
What you will learn
- Replace `for x in sub: yield x` with `yield from sub` when nothing is transformed
- Capture a subgenerator's return value with `result = yield from gen()`
- Know that send(), throw() and close() reach the innermost subgenerator directly
- Write recursive traversals (nested lists, trees) by delegating to yourself
Understanding yield from and delegation
On the surface, `yield from sub` is shorthand for `for item in sub: yield item`: it drains any iterable and re-yields each item to whoever is consuming the outer generator. That alone is worth having, because the loop version is noisier and slightly slower, but it is not the reason `yield from` exists. The real purpose is delegation: while the subgenerator is running, the delegating generator steps out of the way entirely.
The mental model is a chain of frames. When `outer` executes `yield from inner()`, the outer frame suspends and `inner`'s frame is spliced into the chain between the consumer and `outer`. From then on, every value the consumer sends with `send()`, every exception it injects with `throw()`, and a `close()` call all travel straight to `inner`, not to `outer`. A plain `for` loop cannot do this: a value sent into `yield item` lands in the outer frame and is silently dropped, so the subgenerator never sees it.
The second half of delegation is the return channel. A `return value` inside a generator raises `StopIteration(value)`; a `for` loop treats that as "iteration finished" and throws the payload away, but `yield from` catches it and evaluates to `.value`. That single feature turns generators into callable subroutines that both stream results and report a final answer, which is exactly the mechanism `asyncio` coroutines were built on before `async`/`await` existed.
def read_lines(lines):
"""Yield upper-cased lines until a blank one; return how many were read."""
count = 0
for line in lines:
if not line:
return count
count += 1
yield line.upper()
return count
def document(lines):
it = iter(lines) # one shared iterator
header_count = yield from read_lines(it) # return value captured
yield f"--- header had {header_count} lines ---"
body_count = yield from read_lines(it) # resumes after the blank
yield f"--- body had {body_count} lines ---"
data = ["title", "author", "", "para one", "para two", "para three", ""]
for item in document(data):
print(item)yield from hands the entire generator protocol — items out, values in, exceptions, close, and the return value — over to a subiterator.
Worked examples
Recursive flattening
Delegating to itself lets one generator walk an arbitrarily nested structure without an explicit stack.
def flatten(obj):
for item in obj:
if isinstance(item, (list, tuple)):
yield from flatten(item)
else:
yield item
nested = [1, [2, [3, [4, 5]], 6], (7, 8)]
print(list(flatten(nested)))Example explained
Line 1`yield from flatten(item)` creates a new generator one level deeper and forwards all of its items outward.
Line 2Each recursive call peels off exactly one level, so nesting depth becomes call depth, not loop nesting.
Line 3The `else: yield item` branch is a plain yield because a leaf is a single value, not an iterable to drain.
Line 4Strings are deliberately not in the isinstance check, or 'ab' would recurse forever into 'a', 'a', 'a'.
Values sent through the delegation
send() passes straight to the subgenerator, and its return value becomes the result of the yield from expression.
def accumulator():
total = 0
while True:
n = yield total
if n is None:
return total
total += n
def driver():
result = yield from accumulator()
yield f"final: {result}"
g = driver()
print(next(g))
print(g.send(5))
print(g.send(7))
print(g.send(None))Example explained
Line 1`next(g)` starts `driver`, which immediately starts `accumulator` and suspends at `yield total`, so 0 comes out.
Line 2`g.send(5)` is delivered to `accumulator`'s `yield` expression, not to `driver`; `driver` is frozen the whole time.
Line 3`g.send(None)` makes `n` be None, so `accumulator` returns 12 as `StopIteration(12)`.
Line 4`yield from` intercepts that StopIteration, binds `result = 12`, and only then does `driver` run again.
Why a forwarding loop is not equivalent
The same subgenerator behaves differently behind yield from and behind a hand-written for loop.
def echo():
while True:
received = yield
print("sub got", repr(received))
def delegated():
yield from echo()
def manual():
for value in echo():
yield value
print("with yield from:")
g = delegated()
next(g)
g.send("hello")
print("with a for loop:")
h = manual()
next(h)
h.send("hello")Example explained
Line 1In `delegated`, "hello" is handed to `echo`'s own `yield`, so `received` is the string.
Line 2In `manual`, "hello" resumes `yield value` inside `manual`; the result is not assigned anywhere, so it vanishes.
Line 3The loop then calls `next()` on `echo`, which is equivalent to sending None — hence `sub got None`.
Line 4This is why any generator meant to receive data must be reached with `yield from`, never with a relay loop.
Important notes
`yield from` accepts any iterable, but if you `send()` a non-None value while the generator is delegating to a plain iterator such as a list, Python raises AttributeError because lists have no `send()`.
`yield from` is a pass-through only: it cannot filter or transform items. If you need to change what comes out, keep an explicit `for` loop. Inside `async def` it is a SyntaxError — use `await` or `async for` there.
Common mistakes
Writing `total = yield from [1, 2, 3]` and expecting a number: a list iterator has no return value, so `yield from` evaluates to None and the bug shows up much later.
Assuming a subgenerator's `return value` appears in the output stream. It does not — if no caller writes `x = yield from ...`, the value is discarded along with the StopIteration.
Passing the container instead of one shared iterator: `yield from read(data)` twice restarts at element 0, silently re-yielding items you already consumed.
Try it yourself
Change, predict, then run
Write a generator `paths(d, prefix="")` that walks a nested dict and yields `(dotted_key, value)` pairs, delegating with `yield from` when a value is itself a dict. Confirm that `list(paths({"a": 1, "b": {"c": 2, "d": {"e": 3}}}))` gives `[('a', 1), ('b.c', 2), ('b.d.e', 3)]`.
Open the Python workspaceCheck your understanding
Why is `yield from sub` not merely a shorthand for `for x in sub: yield x`?
- Because sent values, thrown exceptions, close() and sub's return value are routed through it, all of which the loop loses
- Because `yield from` is lazy while the `for` loop version materialises the subgenerator first
- Because `yield from` can iterate objects that do not implement `__iter__`
- Because `yield from` automatically flattens nested iterables to any depth
Show answer
The loop reproduces only the outward flow of items; the delegation protocol (send/throw/close reaching the inner frame, plus catching StopIteration.value) is what `yield from` adds. Option 3 is tempting but wrong: `yield from` goes exactly one level deep — flattening comes from calling the generator recursively, not from the keyword. Both forms are equally lazy, and both require an iterable.