PYTHON / ADVANCED PYTHON
functools: cache, partial, and reduce
Use functools.cache/lru_cache to memoize pure functions, partial to pre-bind arguments, and reduce to fold a sequence into one value.
What you will learn
- Memoize a recursive function with @cache and read hits/misses from cache_info()
- Bound memory with @lru_cache(maxsize=N) and reset state with cache_clear()
- Pre-fill arguments with partial, knowing positionals bind from the left
- Fold a sequence with reduce and always supply an initial value
Understanding functools: cache, partial, and reduce
functools.cache wraps a function in a dictionary whose key is built from the call arguments: the positional tuple plus the sorted keyword pairs. That is why every argument must be hashable, and why the cache is only correct for functions that are deterministic and depend on nothing but their arguments. cache is exactly lru_cache(maxsize=None), so it never evicts anything and never does LRU bookkeeping; lru_cache(maxsize=N) keeps only the N most recently used keys and discards the rest.
partial(func, *args, **kwargs) returns a callable object that stores the original function and the arguments you supplied. When you call it, the stored positional arguments are placed before the ones you pass at call time, and the stored keywords are merged with (and overridable by) the call-time keywords. So partial fills parameters from the left, which is the opposite of what most people assume the first time they use it, and the reason a partial object exposes .func, .args, and .keywords for inspection.
reduce(function, iterable, initial) collapses an iterable to a single value by threading an accumulator through a two-argument function: acc = function(acc, item) for each item. Without an initial value the first item becomes the accumulator, which means an empty iterable raises TypeError and a one-item iterable never calls the function at all. Python keeps reduce in functools rather than builtins because sum, min, max, any, all, and math.prod already cover the common folds more readably; reduce earns its place only when the combining step is genuinely custom.
from functools import cache, partial, reduce
calls = 0
cache
def fib(n):
global calls
calls += 1
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(30), calls)
print(fib.cache_info())
from_hex = partial(int, base=16)
print(from_hex("ff"), from_hex("10"))
print(reduce(lambda acc, ch: acc * 2 + int(ch), "10110", 0))cache keys results by hashable arguments, partial pre-binds arguments from the left, and reduce threads an accumulator through a two-argument function.
Worked examples
partial binds from the left
Shows that stored positional arguments come first, stored keywords can be overridden, and the partial object is inspectable.
from functools import partial
def clamp(lo, hi, x):
return max(lo, min(hi, x))
byte = partial(clamp, 0, 255)
print(byte(300), byte(-4))
print(byte.func.__name__, byte.args, byte.keywords)
greet = partial(print, "hi", sep="-")
greet("there", "you")
greet("there", sep="/")Example explained
Line 1partial(clamp, 0, 255) fixes lo and hi because positional arguments fill parameters left to right, leaving x for the call site.
Line 2byte(-4) returns 0: min(255, -4) is -4, then max(0, -4) is 0.
Line 3byte.func, byte.args, and byte.keywords let you see exactly what was pre-bound, which plain lambdas cannot show.
Line 4greet("there", sep="/") proves stored keywords are defaults only: a call-time keyword replaces sep="-".
lru_cache eviction and shared mutable results
Demonstrates that a cache hit returns the identical object, and that maxsize causes recomputation after eviction.
from functools import lru_cache
lru_cache(maxsize=2)
def load(name):
print("loading", name)
return [name]
a = load("x")
b = load("x")
print(a is b)
a.append("mutated")
print(load("x"))
load("y")
load("z")
print(load("x"))Example explained
Line 1The second load("x") prints nothing: the body never runs, and a is b is True because the cache stores the object, not a copy.
Line 2Appending to a mutates the cached list, so the next cache hit returns ['x', 'mutated'] to every caller.
Line 3With maxsize=2, caching y and z evicts the least recently used key x.
Line 4The final load("x") is a miss, so the body runs again and returns a fresh ['x'] with the mutation gone.
reduce with and without an initial value
Shows a custom fold, a fold using operator, and why the initial value matters for possibly-empty input.
from functools import reduce
import operator
rows = [{"a": 1}, {"b": 2}, {"a": 9}]
print(reduce(lambda acc, d: {**acc, **d}, rows, {}))
print(reduce(operator.mul, range(1, 6)))
try:
reduce(operator.add, [])
except TypeError:
print("TypeError: empty input needs an initial value")
print(reduce(operator.add, [], 0))Example explained
Line 1The {} initial value gives the fold a starting accumulator, so later dicts overwrite earlier keys: 'a' ends as 9 but keeps its original position.
Line 2operator.mul over range(1, 6) multiplies 1 through 5 to 120, the same fold that math.prod does natively.
Line 3With no initial value and no items, reduce has nothing to return and raises TypeError instead of guessing a zero value.
Line 4Passing 0 as initial makes the empty case return 0 without any call to operator.add.
Important notes
@cache is unbounded, so caching a method keeps self alive in the cache key and can pin whole objects in memory for the process lifetime.
functools.cache requires Python 3.9 or later; lru_cache(maxsize=None) is the identical, older equivalent.
Common mistakes
Decorating a function that takes a list or dict argument with @cache: it works at import time and raises TypeError: unhashable type: 'list' on the first call.
Caching a function that returns a mutable object and then mutating the result, which silently corrupts the value every later caller receives.
Writing partial(div, 2) expecting 2 to become the divisor: positionals fill from the left, so div(2, 8) runs and you get 0.25 instead of 4.0.
Caching a function that reads a file, clock, or database and wondering why the value never changes; only cache_clear() or a maxsize eviction refreshes it.
Try it yourself
Change, predict, then run
Write a recursive count_paths(r, c) that returns the number of monotone lattice paths across an r-by-c grid, decorate it with @cache, and print count_paths(12, 12) together with count_paths.cache_info(). Then use reduce to compute the product of the digits of 987654 and confirm it matches a plain for loop.
Open the Python workspaceCheck your understanding
Given def div(a, b): return a / b and half = partial(div, 2), what does half(8) return and why?
- 4.0, because partial fills the last unbound parameter
- 0.25, because partial's stored positional goes first, so a=2 and b=8
- TypeError, because partial cannot bind positional arguments
- 2.0, because partial ignores the call-time argument
Show answer
partial prepends its stored positional arguments, so the call becomes div(2, 8) and returns 0.25. The 4.0 answer assumes partial fills parameters from the right; to fix b instead you need partial(div, b=8) or a keyword-based wrapper.