Aggregations, the axis Parameter, Cumulative and NaN-safe Functions

An aggregation turns many values into one: the total, the average, the biggest. The axis parameter decides which direction to aggregate in a 2D or 3D array: per column, per row, or everything. This lesson also covers cumulative functions (running totals) and NaN-safe functions that ignore missing values.

How to run the examples

Create a new file named aggregations_axis.py in your NumpyTutorials folder (see the setup in lesson 01). Any name works; this one just matches the topic. Start the file with these lines:

import numpy as np

Then copy the examples of this page into the same file, one after another, in the order shown, and run it with python aggregations_axis.py (with your virtual environment active). Some examples use variables created in earlier ones, so keep everything in the one file. Each Output box shows what that example prints.

1. Basic aggregations

marks = np.array([72, 85, 64, 91, 58, 77, 85])

print("sum    :", marks.sum())
print("mean   :", marks.mean().round(2))
print("median :", np.median(marks))
print("min/max:", marks.min(), marks.max())
print("range  :", np.ptp(marks), "(max - min)")
print("std    :", marks.std().round(2), "(how spread out)")
print("var    :", marks.var().round(2))
print("np.sum(marks) is the same as marks.sum():", np.sum(marks) == marks.sum())

sum    : 532
mean   : 76.0
median : 77.0
min/max: 58 91
range  : 33 (max - min)
std    : 11.16 (how spread out)
var    : 124.57
np.sum(marks) is the same as marks.sum(): True

Most of these exist both as a method (marks.sum()) and as a function (np.sum(marks)); they give the same result. median is only a function.

2. argmin and argmax

max() tells you what the biggest value is; argmax() tells you where it is. The position lets you look up related data, like the student's name.

names = np.array(["Asha", "Ravi", "Neha", "Kiran", "Meena", "Arjun", "Sneha"])
print("argmax:", marks.argmax(), "->", names[marks.argmax()], "scored", marks.max())
print("argmin:", marks.argmin(), "->", names[marks.argmin()], "scored", marks.min())
print("ties: argmax gives the FIRST one ->", np.argmax(np.array([5, 9, 9])))

argmax: 3 -> Kiran scored 91
argmin: 4 -> Meena scored 58
ties: argmax gives the FIRST one -> 1

When the biggest value appears more than once, argmax returns the first position.

3. axis=0 and axis=1

This is the part that confuses most beginners, so take it slowly. scores has 3 students (rows) x 4 subjects (columns).

# 3 students x 4 subjects
scores = np.array([[78, 85, 92, 66],
                   [88, 79, 95, 70],
                   [65, 90, 81, 99]])

print("scores:\n", scores)
print("sum()        :", scores.sum(), "(everything)")
print("sum(axis=0)  :", scores.sum(axis=0), "-> one per subject (column)")
print("sum(axis=1)  :", scores.sum(axis=1), "-> one per student (row)")
print("max(axis=0)  :", scores.max(axis=0), "-> best mark per subject")
print("argmax(axis=1):", scores.argmax(axis=1), "-> best subject of each student")
print("mean(axis=1, keepdims=True) shape:", scores.mean(axis=1, keepdims=True).shape)

scores:
 [[78 85 92 66]
 [88 79 95 70]
 [65 90 81 99]]
sum()        : 988 (everything)
sum(axis=0)  : [231 254 268 235] -> one per subject (column)
sum(axis=1)  : [321 332 335] -> one per student (row)
max(axis=0)  : [88 90 95 99] -> best mark per subject
argmax(axis=1): [2 2 3] -> best subject of each student
mean(axis=1, keepdims=True) shape: (3, 1)

Easy rule: the axis you name is the one that disappears. Shape (3, 4) with axis=0 loses the 3, so you get 4 results; with axis=1 it loses the 4, so you get 3 results.

keepdims=True keeps the collapsed axis with size 1: (3, 1) instead of (3,). That is exactly the column shape you need to broadcast back against the table (lesson 10).

4. axis on a 3D array

# 2 stores x 3 months x 4 products
data = np.arange(1, 25).reshape(2, 3, 4)
print("shape          :", data.shape)
print("sum(axis=0) -> shape", data.sum(axis=0).shape, "(months x products, stores added)")
print("sum(axis=(1, 2)) -> per store:", data.sum(axis=(1, 2)))
print("sum(axis=2) -> per store and month:\n", data.sum(axis=2))

shape          : (2, 3, 4)
sum(axis=0) -> shape (3, 4) (months x products, stores added)
sum(axis=(1, 2)) -> per store: [ 78 222]
sum(axis=2) -> per store and month:
 [[10 26 42]
 [58 74 90]]

• Shape (2, 3, 4) = stores x months x products. axis=0 adds the stores together, leaving (3, 4).
• A tuple of axes, axis=(1, 2), collapses several axes at once: one total per store.
• axis=2 adds up the products, leaving a store x month table.

5. cumsum, cumprod, diff

Cumulative functions keep every step of the calculation instead of just the final answer.

daily_sales = np.array([100, 150, 80, 200, 170])
print("daily       :", daily_sales)
print("cumsum      :", daily_sales.cumsum(), "(running total)")
print("diff        :", np.diff(daily_sales), "(change from the day before)")

growth = np.array([1.10, 1.05, 0.95, 1.20])          # +10%, +5%, -5%, +20%
print("cumprod     :", growth.cumprod().round(4), "(total growth so far)")
print("running max :", np.maximum.accumulate(np.array([3, 7, 5, 9, 4])))
print("cumsum(axis=1) per student:\n", scores.cumsum(axis=1))

daily       : [100 150  80 200 170]
cumsum      : [100 250 330 530 700] (running total)
diff        : [ 50 -70 120 -30] (change from the day before)
cumprod     : [1.1    1.155  1.0973 1.3167] (total growth so far)
running max : [3 7 7 9 9]
cumsum(axis=1) per student:
 [[ 78 163 255 321]
 [ 88 167 262 332]
 [ 65 155 236 335]]

• cumsum: running total. The last value is the same as sum().
• np.diff: the opposite idea, the change from each value to the next (one value fewer).
• cumprod: running product. Growth factors of +10%, +5%, -5%, +20% give a total of 1.3167 (+31.7%) after four periods.
• np.maximum.accumulate: the running maximum ("highest so far"). Every ufunc has an .accumulate.
• Cumulative functions take axis too: cumsum(axis=1) keeps a running total along each row.

6. NaN-safe functions

np.nan marks a missing value (lesson 14). A single nan spoils normal aggregations: anything plus nan is nan. The nan... versions simply skip missing values.

temps = np.array([31.5, 33.0, np.nan, 29.5, np.nan, 35.0])     # nan = missing reading

print("sum     :", temps.sum(), "   <- one nan makes the result nan")
print("nansum  :", np.nansum(temps))
print("nanmean :", np.nanmean(temps).round(2))
print("nanmin  :", np.nanmin(temps), " nanmax:", np.nanmax(temps))
print("nanstd  :", np.nanstd(temps).round(2))
print("nanargmax:", np.nanargmax(temps))
print("nancumsum:", np.nancumsum(temps))
print("readings that are not nan:", np.count_nonzero(~np.isnan(temps)))

sum     : nan    <- one nan makes the result nan
nansum  : 129.0
nanmean : 32.25
nanmin  : 29.5  nanmax: 35.0
nanstd  : 2.02
nanargmax: 5
nancumsum: [ 31.5  64.5  64.5  94.   94.  129. ]
readings that are not nan: 4

nanmean divides by the number of real values (4), not by 6. To count them, use np.count_nonzero(~np.isnan(temps)).

7. More useful aggregations

print("any mark below 60? :", (marks < 60).any())
print("all marks above 50?:", (marks > 50).all())
print("how many above 80  :", np.count_nonzero(marks > 80))
print("25th / 50th / 75th percentile:", np.percentile(marks, [25, 50, 75]))
print("subjects where everyone passed 65:", (scores > 65).all(axis=0))

any mark below 60? : True
all marks above 50?: True
how many above 80  : 3
25th / 50th / 75th percentile: [68. 77. 85.]
subjects where everyone passed 65: [False  True  True  True]

• .any() and .all() on a condition answer yes/no questions about the whole array, or per column with axis=0.
• np.count_nonzero(condition) counts how many elements match.
• np.percentile(marks, 75): 75% of the marks are at or below this value. The 50th percentile is the median.

8. Real example: monthly report of 3 shops

shops = np.array(["Pune", "Mumbai", "Nagpur"])
months = np.array(["Jan", "Feb", "Mar", "Apr"])
revenue = np.array([[120, 135, np.nan, 160],
                    [220, 210, 230, 240],
                    [90, np.nan, 100, 115]])

print("total per shop  :", np.nansum(revenue, axis=1))
print("average / month :", np.nanmean(revenue, axis=0).round(1))
print("best shop       :", shops[np.nansum(revenue, axis=1).argmax()])
print("best month      :", months[np.nansum(revenue, axis=0).argmax()])
print("missing per shop:", np.isnan(revenue).sum(axis=1))
print("running total of Mumbai:", revenue[1].cumsum())

total per shop  : [415. 900. 305.]
average / month : [143.3 172.5 165.  171.7]
best shop       : Mumbai
best month      : Apr
missing per shop: [1 0 1]
running total of Mumbai: [220. 430. 660. 900.]

The revenue table has two missing months. NaN-safe functions with the right axis give totals per shop (axis=1) and averages per month (axis=0) without any cleaning. argmax plus the label arrays name the best shop and month, and np.isnan(...).sum(axis=1) counts the missing months per shop.

Exercises

Try these exercises on your own before moving on:

1. For np.array([[3, 7, 2], [8, 1, 9]]) print the column sums, row means and the position of the biggest value in each row.
2. Daily steps of a week: print the running total and the best day.
3. Replace one value with np.nan and compare mean() with np.nanmean().
4. For a (4, 7) array of 4 weeks x 7 days, find the average of each weekday.

Summary

• sum, mean, median, min, max, std reduce many values to one.
• argmin/argmax give the position of the smallest/biggest value.
• The axis you name disappears: axis=0 gives one result per column, axis=1 one per row.
• keepdims=True keeps the shape ready for broadcasting.
• cumsum, cumprod and diff give running totals and changes.
• Use nansum, nanmean and friends when the data has missing values.

Next: Lesson 12: Sorting, Searching, unique and Sets.