NumPy

In this lesson, we'll learn about ways to represent and manipulate images in Python. By the end of this lesson, students will be able to:

• Apply ndarray arithmetic and logical operators with numbers and other arrays.
• Analyze the shape of an ndarray and index into a multidimensional array.
• Apply arithmetic operators, indexing, and slicing to manipulate RGB images.

We'll need two new modules: imageio, which provides utilities to read and write images in Python, and numpy, which provides the data structures for representing images in Python.

import imageio.v3 as iio
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

Reading an image

Let's use imread to load a color picture of Dubs II in grayscale with mode="L" standing for "luminance" or "lightness". To show an image, we can plot its pixels using the matplotlib function imshow.

dubs = iio.imread("dubs.jpg", mode="L")
plt.imshow(dubs, cmap="gray")

<matplotlib.image.AxesImage at 0x79453d8b0af0>

Pandas uses NumPy to represent a Series of values, so many element-wise operations should seem familiar. In fact, we can load an image into a Pandas DataFrame and see that this grayscale image is really a 2-d array of color values ranging from [0, 255].

pd.DataFrame(dubs)

	0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
0	238	238	239	239	240	240	241	241	240	240	...	231	231	232	232	232	232	232	232	232	232
1	238	239	239	239	240	240	241	241	240	240	...	231	231	232	232	232	232	232	232	232	232
2	239	239	239	240	240	240	240	241	240	240	...	231	231	232	232	232	232	232	232	232	232
3	239	239	240	240	240	240	240	241	240	240	...	231	231	232	232	232	232	232	232	232	232
4	240	240	240	240	240	240	240	240	240	240	...	231	231	232	232	232	232	232	232	232	232
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
355	240	240	240	240	240	240	240	240	240	240	...	237	237	237	237	237	237	237	237	237	237
356	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236
357	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236
358	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236
359	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236

360 rows × 640 columns

What would a color image of Dubs II look like instead? Let's try loading the picture without mode="L" to maintain its color data.

dubs = iio.imread("dubs.jpg")
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d954ac0>

What do you think the colorful DataFrame should look like?

pd.DataFrame(dubs)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[25], line 1
----> 1 pd.DataFrame(dubs)

File /opt/conda/lib/python3.10/site-packages/pandas/core/frame.py:782, in DataFrame.__init__(self, data, index, columns, dtype, copy)
    771         mgr = dict_to_mgr(
    772             # error: Item "ndarray" of "Union[ndarray, Series, Index]" has no
    773             # attribute "name"
   (...)
    779             copy=_copy,
    780         )
    781     else:
--> 782         mgr = ndarray_to_mgr(
    783             data,
    784             index,
    785             columns,
    786             dtype=dtype,
    787             copy=copy,
    788             typ=manager,
    789         )
    791 # For data is list-like, or Iterable (will consume into list)
    792 elif is_list_like(data):

File /opt/conda/lib/python3.10/site-packages/pandas/core/internals/construction.py:314, in ndarray_to_mgr(values, index, columns, dtype, copy, typ)
    308     _copy = (
    309         copy_on_sanitize
    310         if (dtype is None or astype_is_view(values.dtype, dtype))
    311         else False
    312     )
    313     values = np.array(values, copy=_copy)
--> 314     values = _ensure_2d(values)
    316 else:
    317     # by definition an array here
    318     # the dtypes will be coerced to a single dtype
    319     values = _prep_ndarraylike(values, copy=copy_on_sanitize)

File /opt/conda/lib/python3.10/site-packages/pandas/core/internals/construction.py:592, in _ensure_2d(values)
    590     values = values.reshape((values.shape[0], 1))
    591 elif values.ndim != 2:
--> 592     raise ValueError(f"Must pass 2-d input. shape={values.shape}")
    593 return values

ValueError: Must pass 2-d input. shape=(360, 640, 3)

shape=(360, 640, 3) indicates 360 pixels high, 640 pixels wide, and 3 color channels: red, green, blue.

# How do I get all the red color pixels as a 360x640 array?
pd.DataFrame(dubs[:, :, 0])

	0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
0	235	235	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
1	235	236	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
2	236	236	236	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
3	236	236	237	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
4	237	237	237	237	237	237	237	237	237	237	...	228	228	229	229	229	229	229	229	229	229
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
355	237	237	237	237	237	237	237	237	237	237	...	234	234	234	234	234	234	234	234	234	234
356	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233
357	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233
358	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233
359	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233

360 rows × 640 columns

# How do I get all the green color pixels as a 360x640 array?
pd.DataFrame(dubs[:, :, 1])

	0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
0	238	238	239	239	240	240	241	241	240	240	...	230	230	231	231	231	231	231	231	231	231
1	238	239	239	239	240	240	241	241	240	240	...	230	230	231	231	231	231	231	231	231	231
2	239	239	239	240	240	240	240	241	240	240	...	230	230	231	231	231	231	231	231	231	231
3	239	239	240	240	240	240	240	241	240	240	...	230	230	231	231	231	231	231	231	231	231
4	240	240	240	240	240	240	240	240	240	240	...	230	230	231	231	231	231	231	231	231	231
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
355	240	240	240	240	240	240	240	240	240	240	...	236	236	236	236	236	236	236	236	236	236
356	240	240	240	240	240	240	240	240	240	240	...	236	236	235	235	235	235	235	235	235	235
357	240	240	240	240	240	240	240	240	240	240	...	236	236	235	235	235	235	235	235	235	235
358	240	240	240	240	240	240	240	240	240	240	...	236	236	235	235	235	235	235	235	235	235
359	240	240	240	240	240	240	240	240	240	240	...	236	236	235	235	235	235	235	235	235	235

360 rows × 640 columns

# How do I get all the blue color pixels as a 360x640 array?
pd.DataFrame(dubs[:, :, 2])

	0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
0	245	245	246	246	247	247	248	248	247	247	...	242	242	243	243	243	243	243	243	243	243
1	245	246	246	246	247	247	248	248	247	247	...	242	242	243	243	243	243	243	243	243	243
2	246	246	246	247	247	247	247	248	247	247	...	242	242	243	243	243	243	243	243	243	243
3	246	246	247	247	247	247	247	248	247	247	...	242	242	243	243	243	243	243	243	243	243
4	247	247	247	247	247	247	247	247	247	247	...	242	242	243	243	243	243	243	243	243	243
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
355	247	247	247	247	247	247	247	247	247	247	...	248	248	248	248	248	248	248	248	248	248
356	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
357	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
358	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
359	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247

360 rows × 640 columns

# How do I get a DataFrame representing all the pixels in all the colors?
pd.concat([
    pd.DataFrame(dubs[:, :, 0]),
    pd.DataFrame(dubs[:, :, 1]),
    pd.DataFrame(dubs[:, :, 2])
], keys=["R", "G", "B"])

		0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
R	0	235	235	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	1	235	236	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	2	236	236	236	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	3	236	236	237	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	4	237	237	237	237	237	237	237	237	237	237	...	228	228	229	229	229	229	229	229	229	229
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
B	355	247	247	247	247	247	247	247	247	247	247	...	248	248	248	248	248	248	248	248	248	248
	356	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
	357	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
	358	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
	359	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247

1080 rows × 640 columns

Array manipulation

Images are represented in Python with the type numpy.ndarray or "n-dimensional array." Grayscale images are 2-dimensional arrays with pixel luminance values indicated in each position. Color images are 3-dimensional arrays with pixel color values indicated for each channel (red, green, blue) in each position. Can you set the left and right sides of this picture to 0 so that Dubs II appears surrounded by black borders?

dubs = iio.imread("dubs.jpg")
dubs[:50, :25] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d72c8b0>

dubs = iio.imread("dubs.jpg")
# How to add left and right black-color bars to the picture?
dubs[:, :100] = 0
dubs[:, -100:] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d6dcdf0>

dubs = iio.imread("dubs.jpg")
# Other ways to achieve this result!
dubs[:, :100] = dubs[:, -100:] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d68c1c0>

dubs = iio.imread("dubs.jpg")
# Other ways to achieve this result!
# dubs[:, :100] = dubs[:, -100:] = 0
dubs[:, :100, :] = dubs[:, -100:, :] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d560070>

dubs = iio.imread("dubs.jpg")
# Other ways to achieve this result!
# dubs[:, :100] = dubs[:, -100:] = 0
# dubs[:, :100, :] = dubs[:, -100:, :] = 0
dubs[:, :100, :] = dubs[:, -100:, :] = [180, 0, 180]
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d68dde0>

dubs = iio.imread("dubs.jpg")
# Other ways to achieve this result!
# dubs[:, :100] = dubs[:, -100:] = 0
# dubs[:, :100, :] = dubs[:, -100:, :] = 0
dubs[:, :100, :] = dubs[:, -100:, :] = 128
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d4b4940>

When we're performing an assignment on 2-dimensions of a 3-dimensional image, NumPy follows broadcasting rules to evaluate the operation. The simplest version of broadcasting are just element-wise operations.

dubs = iio.imread("dubs.jpg")
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x79453d2043a0>

Let's try a more complicated example. Using the floor division operator, fill in the imshow call to decrease only the green channel so that the overall picture is much more purple than before.

dubs = iio.imread("dubs.jpg")
# Why does option 3 work as an assignment statement, but not as an expression?
# Why do we slice [:, :, 1] to indicate green when green is specified as a triplet [1, 2, 1]?
dubs[:, :, 1] //= 2
plt.imshow(dubs, cmap="gray")

<matplotlib.image.AxesImage at 0x79453d1286a0>

dubs = iio.imread("dubs.jpg")
# Why isn't this height or width changed by the floor division operation?
# How does NumPy know to only apply the divisions to color channels,
  # and not the other dimensions of the picture?
plt.imshow(dubs // [1, 2, 1])

<matplotlib.image.AxesImage at 0x79453d12b250>

dubs // [:, 2, :] # SyntaxError even before NumPy
# Why is this an error? We're trying to specify a triplet!
  # Python sees that we're trying to slice outside the context of slicing!

  Cell In[59], line 1
    dubs // [:, 2, :] # SyntaxError even before NumPy
             ^
SyntaxError: invalid syntax

[1, 2, 1]

[1, 2, 1]

dubs[:, :, 1]

array([[238, 238, 239, ..., 231, 231, 231],
       [238, 239, 239, ..., 231, 231, 231],
       [239, 239, 239, ..., 231, 231, 231],
       ...,
       [240, 240, 240, ..., 235, 235, 235],
       [240, 240, 240, ..., 235, 235, 235],
       [240, 240, 240, ..., 235, 235, 235]], dtype=uint8)

Practice: Instafade

Write code to apply a fading filter to the image. The fading filter reduces all color values to 77% intensity and then adds 38.25 to each resulting color value. (These numbers are somewhat arbitrarily chosen to get the desired effect.)

The provided code converts the dog array from integer values to floating-point decimal values. To display the final image, the code converts the numbers in the dog array back to uint8 before passing the result to imshow.

dog = iio.imread("dog.jpg").astype("float32")
dog = dog * 0.77 + 38.25
plt.imshow(dog.astype("uint8"))

<matplotlib.image.AxesImage at 0x79453d377430>

Practice: Gotham

Write code to apply the following operations to an image.

1. Expand the red colors by 50% by subtracting 128 from each red channel value, multiply the result by 1.5, and then add 128 to restore the original value range.
2. Increase the blue colors by 13 by adding 13 to each blue channel value.
3. Add black letterboxing bars by setting the top 150 and bottom 150 pixels to black.
4. Clip color values outside the range [0, 255] by reassign all values above 255 to 255 and all values below 0 to 0.

dog = iio.imread("dog.jpg").astype("float32")
# Expand the red colors by 50%
dog[:, :, 0] = (dog[:, :, 0] - 128) * 1.5 + 128
# Increase the blue colors by 13
dog[:, :, 2] += 13
# Add black letterboxing bars
dog[:150] = dog[-150:] = 0
# Clip color values outside the range [0, 255] (there's also an np.clip function)
dog[dog > 255] = 255
dog[dog < 0] = 0
plt.imshow(dog.astype("uint8"))

<matplotlib.image.AxesImage at 0x79453d5d5cc0>

Advanced broadcasting

What is the result of adding the following two arrays together following the broadcasting rules?

x = np.array([[1], [2], [3]])
x

y = np.array([1, 2, 3])
y