Views vs Copies, reshape, ravel and flatten

This lesson is about one of the most important ideas in NumPy, views and copies, and the functions that change an array's shape: reshape, ravel and flatten. They belong together, because reshaping usually gives you a view.

• A view is a new array object that shares the data of the original. Change one and the other changes too.
• A copy has its own data. Changing it never touches the original.

Think of a view as a window into the same room: different window, same furniture. Move a chair through one window and everyone sees it moved. A copy is a photo of the room: you can draw on the photo, but the real room does not change.

How to run the examples

Create a new file named views_copies_reshape.py in your NumpyTutorials folder (see the setup in lesson 01). Any name works; this one just matches the topic. Start the file with these lines:

import numpy as np

Then copy the examples of this page into the same file, one after another, in the order shown, and run it with python views_copies_reshape.py (with your virtual environment active). Some examples use variables created in earlier ones, so keep everything in the one file. Each Output box shows what that example prints.

1. A slice is a view

prices = np.array([100, 200, 300, 400, 500])
middle = prices[1:4]                 # a view of 200, 300, 400
middle[0] = 999                      # change the view ...

print("middle  :", middle)
print("prices  :", prices, "<- the original changed too!")
print("shares memory:", np.shares_memory(prices, middle))
print("middle.base is prices:", middle.base is prices)

middle  : [999 300 400]
prices  : [100 999 300 400 500] <- the original changed too!
shares memory: True
middle.base is prices: True

We changed middle, and prices changed as well, because middle is just a window onto positions 1 to 3 of prices. Two ways to check:

• np.shares_memory(a, b) is True when both arrays use the same memory.
• view.base is the array the view came from (it is None for an array that owns its data).

NumPy does this on purpose: slicing a large array is instant and uses no extra memory, because nothing is copied.

2. copy() gives independent data

prices = np.array([100, 200, 300, 400, 500])
middle = prices[1:4].copy()
middle[0] = 999

print("middle  :", middle)
print("prices  :", prices, "<- not changed")
print("shares memory:", np.shares_memory(prices, middle))

middle  : [999 300 400]
prices  : [100 200 300 400 500] <- not changed
shares memory: False

Add .copy() whenever you want to change a part of an array without affecting the original.

3. Fancy and boolean indexing make copies

prices = np.array([100, 200, 300, 400, 500])
picked = prices[[0, 2]]              # fancy indexing
expensive = prices[prices > 250]     # boolean mask
picked[:] = 0
expensive[:] = 0
print("prices  :", prices, "<- not changed")

# But ASSIGNING through them changes the original:
prices[prices > 250] = 0
print("prices[prices > 250] = 0 ->", prices)

prices  : [100 200 300 400 500] <- not changed
prices[prices > 250] = 0 -> [100 200   0   0   0]

But assigning through a mask or fancy index (prices[prices > 250] = 0) does change the original, because then NumPy writes straight into prices.

4. A common bug

original_marks = np.array([45, 67, 89, 32])
top_two = original_marks[:2]         # meant to be a separate list
top_two += 10                        # oops, changes original_marks
print("original marks:", original_marks)
print("Fix: original_marks[:2].copy()")

original marks: [55 77 89 32]
Fix: original_marks[:2].copy()

The programmer wanted a separate list of the first two marks and added 10 to it, but the original marks 45 and 67 became 55 and 77. In a real program this kind of bug can be very hard to find. The fix is original_marks[:2].copy().

5. reshape

reshape gives the same values a new shape. The total number of elements must stay the same.

monthly = np.arange(1, 13)                   # 12 values
print("monthly        :", monthly)
print("reshape(3, 4)  :\n", monthly.reshape(3, 4))
print("reshape(4, 3)  :\n", monthly.reshape(4, 3))
print("reshape(2, 2, 3):\n", monthly.reshape(2, 2, 3))
print("reshape(-1, 6) :\n", monthly.reshape(-1, 6), "(-1 = work it out)")
print("reshape(12, 1) shape:", monthly.reshape(12, 1).shape)

try:
    monthly.reshape(5, 3)                    # 15 places for 12 values
except ValueError as error:
    print("Error:", error)

grid = monthly.reshape(3, 4)
grid[0, 0] = 100
print("reshape is a view, so monthly changed:", monthly)

monthly        : [ 1  2  3  4  5  6  7  8  9 10 11 12]
reshape(3, 4)  :
 [[ 1  2  3  4]
 [ 5  6  7  8]
 [ 9 10 11 12]]
reshape(4, 3)  :
 [[ 1  2  3]
 [ 4  5  6]
 [ 7  8  9]
 [10 11 12]]
reshape(2, 2, 3):
 [[[ 1  2  3]
  [ 4  5  6]]

 [[ 7  8  9]
  [10 11 12]]]
reshape(-1, 6) :
 [[ 1  2  3  4  5  6]
 [ 7  8  9 10 11 12]] (-1 = work it out)
reshape(12, 1) shape: (12, 1)
Error: cannot reshape array of size 12 into shape (5,3)
reshape is a view, so monthly changed: [100   2   3   4   5   6   7   8   9  10  11  12]

• 12 values can become 3x4, 4x3, 2x2x3, and so on, because each multiplies to 12.
• -1 means "work this number out for me": reshape(-1, 6) becomes 2x6. Only one -1 is allowed.
• reshape(5, 3) needs 15 values, so it fails.
• reshape returns a view when it can: changing grid[0, 0] changed monthly[0] too.

6. ravel vs flatten

Both turn any array into 1D. The difference is view vs copy:

table = np.array([[1, 2, 3],
                  [4, 5, 6]])

flat_view = table.ravel()            # view (when possible)
flat_copy = table.flatten()          # always a copy
print("ravel   :", flat_view)
print("flatten :", flat_copy)

flat_view[0] = 111
flat_copy[1] = 222
print("table after changing both:\n", table, "<- only ravel changed it")

ravel   : [1 2 3 4 5 6]
flatten : [1 2 3 4 5 6]
table after changing both:
 [[111   2   3]
 [  4   5   6]] <- only ravel changed it

7. order='C' vs order='F'

When NumPy reads values in "flat" order, it goes row by row by default. This is called C order, after the C language. order="F" goes column by column (Fortran order).

table = np.array([[1, 2, 3],
                  [4, 5, 6]])
print("ravel()          :", table.ravel())            # row by row (default)
print("ravel(order='F') :", table.ravel(order="F"))   # column by column
print("reshape(3, 2, order='F'):\n", np.arange(6).reshape(3, 2, order="F"))

ravel()          : [1 2 3 4 5 6]
ravel(order='F') : [1 4 2 5 3 6]
reshape(3, 2, order='F'):
 [[0 3]
 [1 4]
 [2 5]]

You rarely need "F", but you will see it in code that works with Fortran or MATLAB data, which stores matrices column by column. Lesson 16 shows how this order is stored in memory.

8. Real example: monthly sales into quarters

sales = np.array([120, 135, 150, 160, 170, 155, 180, 190, 210, 230, 250, 300])
quarters = sales.reshape(4, 3)                # 4 quarters x 3 months
print("quarters:\n", quarters)
print("quarter totals :", quarters.sum(axis=1))
print("best quarter   : Q" + str(quarters.sum(axis=1).argmax() + 1))
print("back to months :", quarters.ravel())

quarters:
 [[120 135 150]
 [160 170 155]
 [180 190 210]
 [230 250 300]]
quarter totals : [405 485 580 780]
best quarter   : Q4
back to months : [120 135 150 160 170 155 180 190 210 230 250 300]

12 monthly values reshaped to 4x3 put each quarter on its own row, so sum(axis=1) gives quarter totals with no loop. ravel() turns the table back into months.

Exercises

Try these exercises on your own before moving on:

1. a = np.arange(10); b = a[::2]. Change b[0]. Does a change? Check with np.shares_memory.
2. Reshape np.arange(24) into (2, 3, 4) and then into (6, -1).
3. Make a 4x5 array, flatten it, change the flat copy and show the 4x5 array did not change.
4. Turn 365 daily values (np.arange(365)) into weeks: why does reshape(-1, 7) fail? Use the first 364 values instead.

Summary

• A view shares data with the original; a copy has its own.
• Slices, reshape, .T and ravel give views; fancy indexing, masks, flatten and .copy() give copies.
• Check with np.shares_memory() or .base; use .copy() when in doubt.
• reshape keeps the number of elements; -1 lets NumPy work out one size.
• NumPy reads values row by row (C order) unless you ask for order="F".

Next: Lesson 07: Transpose, newaxis, expand_dims, squeeze.