PYTHON / PANDAS
Aggregation and named aggregation
Summarise data with .agg using strings, lists, and dicts, and use named aggregation to get flat, predictably ordered output columns.
What you will learn
- Choose between string, callable, list, and dict specs based on the output shape you want
- Write name=(source_column, func) keywords to get single-level, explicitly ordered columns
- Flatten the MultiIndex that list-style agg creates, or avoid it with named aggregation
- Distinguish 'size' (rows per group) from 'count' (non-null values in that column)
Understanding Aggregation and named aggregation
An aggregation is any function that collapses many values into one: sum, mean, max, nunique, or something you write yourself. The .agg method is a dispatcher, and the shape of what you pass decides the shape of what you get back. A single string reduces each column to one number, a list of strings produces one result per function, and a dict lets you say which functions apply to which column. The functions themselves never see the group keys, only the values of one column at a time.
The awkward part of list and dict specs is labelling. Because a number in the result comes from a (column, function) pair, pandas has to record both, so it puts the function name in a second level of the column index for grouped results, or in the row index for whole-frame results. That is why you end up writing g[("units", "sum")] and hand-flattening columns afterwards. Named aggregation inverts the order of the description: you state the output name first, then the source column and the reduction that feeds it, so the result has a single level of columns in exactly the order you typed the keywords.
Prefer string names over lambdas when a string exists. Strings like "sum", "mean", "size" and "nunique" are routed to pandas' internal group-aware implementations, while a callable is invoked once per group per column in Python; a list of lambdas also forces pandas to invent names like <lambda_0>. Whatever you pass must return one scalar per group, since the point of aggregation is that each group becomes exactly one row.
import pandas as pd
sales = pd.DataFrame({
"region": ["north", "south", "north", "south", "north"],
"rep": ["ana", "bo", "cai", "dee", "ana"],
"units": [12, 30, 7, 22, 19],
"price": [4.5, 2.0, 4.5, 2.5, 4.0],
})
# dict spec: function names become the row labels, columns stay as they were
print(sales.agg({"units": ["sum", "mean"], "price": ["min", "max"]}))
print()
# named aggregation: one keyword per output column
summary = sales.groupby("region").agg(
total_units=("units", "sum"),
orders=("units", "size"),
best_price=("price", "max"),
avg_units=("units", "mean"),
)
print(summary)The specification you hand to .agg determines the labelling of the result, and the name=(column, function) keyword form is the one that yields flat, self-documenting columns.
Worked examples
What a list spec actually returns
Shows the two-level columns and the mangled lambda name produced by a list of functions, then flattens them by hand.
import pandas as pd
scores = pd.DataFrame({
"class": ["a", "a", "b", "b", "b"],
"score": [70, 90, 55, 65, 80],
})
multi = scores.groupby("class").agg(
{"score": ["min", "max", lambda s: s.max() - s.min()]}
)
print(list(multi.columns))
multi.columns = ["min", "max", "spread"]
print(multi)Example explained
Line 1The dict-of-lists spec builds a MultiIndex on columns: level 0 is the source column, level 1 the function label.
Line 2A lambda has no meaningful __name__, so pandas renames it <lambda_0> based on its position in the list.
Line 3Assigning to .columns replaces the MultiIndex with plain strings; named aggregation gets you here in one step.
Line 4The spread values 20 and 25 confirm the lambda received each group's score values as a Series.
pd.NamedAgg with a custom function
Uses the explicit pd.NamedAgg spelling and a user-defined reducer alongside built-in string aggregators.
import pandas as pd
orders = pd.DataFrame({
"customer": ["x", "y", "x", "y", "x"],
"amount": [10.0, 25.0, 30.0, 8.0, 15.0],
})
def spread(s):
return s.max() - s.min()
out = orders.groupby("customer").agg(
n=pd.NamedAgg(column="amount", aggfunc="count"),
total=pd.NamedAgg(column="amount", aggfunc="sum"),
swing=pd.NamedAgg(column="amount", aggfunc=spread),
)
print(out)Example explained
Line 1pd.NamedAgg(column=..., aggfunc=...) is the same thing as the two-tuple, just written out for readability.
Line 2spread is called once per group with that group's amount Series and returns a single float, which is what agg requires.
Line 3Output columns appear in keyword order (n, total, swing), not in the order pandas happens to compute them.
Line 4The group key becomes the index and keeps the name customer, so nothing about the source column names leaks into the columns.
Aggregating a whole frame, no grouping
Contrasts a list spec (one row per function) with a single string spec (a Series indexed by column name).
import pandas as pd
df = pd.DataFrame({"units": [12, 30, 7], "price": [4.5, 2.0, 4.5]})
print(df.agg(["sum", "mean"]))
print()
print(df.agg("sum"))Example explained
Line 1Without groupby there is nothing to key on, so the function names go into the row index instead of the columns.
Line 2units now holds an integer-valued sum and a fractional mean in one column, so the column is float64 and both values share the six-decimal format.
Line 3df.agg("sum") reduces every column to one scalar and returns a Series labelled by column name, exactly what df.sum() does.
Line 4That Series mixes an int64 sum with a float64 sum, so it is upcast and 49 prints as 49.0.
Important notes
"size" counts every row in the group while "count" counts only non-null values of that column, so the two diverge the moment data is missing.
Named aggregation output names are Python keyword arguments, so they must be valid identifiers; a label like "total units" or "2024" needs the dict form followed by a rename.
Common mistakes
Writing agg({"units": "sum", "units": "mean"}): Python collapses the duplicate dict key before pandas sees it, so you silently get only the mean. Use {"units": ["sum", "mean"]} or two named keywords.
Passing a computed value instead of a function, as in agg(total=("units", sales["units"].sum())): the second element must be a function or its string name, so pandas raises a TypeError instead of aggregating.
Dropping the tuple, as in agg(total="sum"): named aggregation cannot guess the source column and raises a TypeError asking for (column, aggfunc) pairs.
Try it yourself
Change, predict, then run
Create a DataFrame of five trips with columns city, minutes and fare, then use named aggregation to produce one row per city with n_trips (row count), total_fare (sum of fare) and longest (max minutes), and sort the result by total_fare descending.
Open the Python workspaceCheck your understanding
You run g = sales.groupby("region").agg({"units": ["sum", "mean"]}) and then g["units_sum"] raises KeyError. What is going on?
- The dict-of-lists form labels results as (column, function) pairs, so the sum lives at g[("units", "sum")]; flat names like units_sum come from named aggregation
- agg silently dropped the sum because a column can only be aggregated by one function per call
- You must call reset_index() before any column of a grouped result can be accessed
- String function names are not allowed inside a dict, so nothing was computed and the frame is empty
Show answer
A list of functions forces pandas to record both the source column and the function, which it does with a two-level column index, so the label is the tuple ("units", "sum") and no key named units_sum exists. reset_index() does not help: it only moves the region key from the index into a column and leaves the two-level column labels exactly as they are.