NumPy

In this lesson, we'll learn about ways to represent and manipulate images in Python. By the end of this lesson, students will be able to:

• Apply ndarray arithmetic and logical operators with numbers and other arrays.
• Analyze the shape of an ndarray and index into a multidimensional array.
• Apply arithmetic operators, indexing, and slicing to manipulate RGB images.

We'll need two new modules: imageio, which provides utilities to read and write images in Python, and numpy, which provides the data structures for representing images in Python.

import imageio.v3 as iio
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

Reading an image

Let's use imread to load a color picture of Dubs II in grayscale with mode="L" standing for "luminance" or "lightness". To show an image, we can plot its pixels using the matplotlib function imshow.

dubs = iio.imread("dubs.jpg", mode="L")
plt.imshow(dubs, cmap="gray")

<matplotlib.image.AxesImage at 0x7ec7cdd00a00>

dubs.ndim

2

dubs.shape

(360, 640)

plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7dc8bcd60>

dubs = iio.imread("dubs.jpg")
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cef17700>

dubs

array([[[235, 238, 245],
        [235, 238, 245],
        [236, 239, 246],
        ...,
        [229, 231, 243],
        [229, 231, 243],
        [229, 231, 243]],

       [[235, 238, 245],
        [236, 239, 246],
        [236, 239, 246],
        ...,
        [229, 231, 243],
        [229, 231, 243],
        [229, 231, 243]],

       [[236, 239, 246],
        [236, 239, 246],
        [236, 239, 246],
        ...,
        [229, 231, 243],
        [229, 231, 243],
        [229, 231, 243]],

       ...,

       [[237, 240, 247],
        [237, 240, 247],
        [237, 240, 247],
        ...,
        [233, 235, 247],
        [233, 235, 247],
        [233, 235, 247]],

       [[237, 240, 247],
        [237, 240, 247],
        [237, 240, 247],
        ...,
        [233, 235, 247],
        [233, 235, 247],
        [233, 235, 247]],

       [[237, 240, 247],
        [237, 240, 247],
        [237, 240, 247],
        ...,
        [233, 235, 247],
        [233, 235, 247],
        [233, 235, 247]]], dtype=uint8)

Pandas uses NumPy to represent a Series of values, so many element-wise operations should seem familiar. In fact, we can load an image into a Pandas DataFrame and see that this grayscale image is really a 2-d array of color values ranging from [0, 255].

pd.DataFrame(dubs)

	0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
0	238	238	239	239	240	240	241	241	240	240	...	231	231	232	232	232	232	232	232	232	232
1	238	239	239	239	240	240	241	241	240	240	...	231	231	232	232	232	232	232	232	232	232
2	239	239	239	240	240	240	240	241	240	240	...	231	231	232	232	232	232	232	232	232	232
3	239	239	240	240	240	240	240	241	240	240	...	231	231	232	232	232	232	232	232	232	232
4	240	240	240	240	240	240	240	240	240	240	...	231	231	232	232	232	232	232	232	232	232
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
355	240	240	240	240	240	240	240	240	240	240	...	237	237	237	237	237	237	237	237	237	237
356	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236
357	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236
358	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236
359	240	240	240	240	240	240	240	240	240	240	...	237	237	236	236	236	236	236	236	236	236

360 rows × 640 columns

What would a color image of Dubs II look like instead? Let's try loading the picture without mode="L" to maintain its color data.

dubs = iio.imread("dubs.jpg")
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cde7b3a0>

What do you think the colorful DataFrame should look like?

dubs.ndim

3

pd.DataFrame(dubs)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[10], line 1
----> 1 pd.DataFrame(dubs)

File /opt/conda/lib/python3.10/site-packages/pandas/core/frame.py:782, in DataFrame.__init__(self, data, index, columns, dtype, copy)
    771         mgr = dict_to_mgr(
    772             # error: Item "ndarray" of "Union[ndarray, Series, Index]" has no
    773             # attribute "name"
   (...)
    779             copy=_copy,
    780         )
    781     else:
--> 782         mgr = ndarray_to_mgr(
    783             data,
    784             index,
    785             columns,
    786             dtype=dtype,
    787             copy=copy,
    788             typ=manager,
    789         )
    791 # For data is list-like, or Iterable (will consume into list)
    792 elif is_list_like(data):

File /opt/conda/lib/python3.10/site-packages/pandas/core/internals/construction.py:314, in ndarray_to_mgr(values, index, columns, dtype, copy, typ)
    308     _copy = (
    309         copy_on_sanitize
    310         if (dtype is None or astype_is_view(values.dtype, dtype))
    311         else False
    312     )
    313     values = np.array(values, copy=_copy)
--> 314     values = _ensure_2d(values)
    316 else:
    317     # by definition an array here
    318     # the dtypes will be coerced to a single dtype
    319     values = _prep_ndarraylike(values, copy=copy_on_sanitize)

File /opt/conda/lib/python3.10/site-packages/pandas/core/internals/construction.py:592, in _ensure_2d(values)
    590     values = values.reshape((values.shape[0], 1))
    591 elif values.ndim != 2:
--> 592     raise ValueError(f"Must pass 2-d input. shape={values.shape}")
    593 return values

ValueError: Must pass 2-d input. shape=(360, 640, 3)

# red color channel of the rgb image
dubs[:, :, 0]

array([[235, 235, 236, ..., 229, 229, 229],
       [235, 236, 236, ..., 229, 229, 229],
       [236, 236, 236, ..., 229, 229, 229],
       ...,
       [237, 237, 237, ..., 233, 233, 233],
       [237, 237, 237, ..., 233, 233, 233],
       [237, 237, 237, ..., 233, 233, 233]], dtype=uint8)

pd.DataFrame(dubs[:, :, 0])

	0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
0	235	235	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
1	235	236	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
2	236	236	236	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
3	236	236	237	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
4	237	237	237	237	237	237	237	237	237	237	...	228	228	229	229	229	229	229	229	229	229
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
355	237	237	237	237	237	237	237	237	237	237	...	234	234	234	234	234	234	234	234	234	234
356	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233
357	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233
358	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233
359	237	237	237	237	237	237	237	237	237	237	...	234	234	233	233	233	233	233	233	233	233

360 rows × 640 columns

dubs.shape

(360, 640, 3)

pd.concat([
    pd.DataFrame(dubs[:, :, 0]),
    pd.DataFrame(dubs[:, :, 1]),
    pd.DataFrame(dubs[:, :, 2])
], keys=['R', 'G', 'B'])

		0	1	2	3	4	5	6	7	8	9	...	630	631	632	633	634	635	636	637	638	639
R	0	235	235	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	1	235	236	236	236	237	237	238	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	2	236	236	236	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	3	236	236	237	237	237	237	237	238	237	237	...	228	228	229	229	229	229	229	229	229	229
	4	237	237	237	237	237	237	237	237	237	237	...	228	228	229	229	229	229	229	229	229	229
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
B	355	247	247	247	247	247	247	247	247	247	247	...	248	248	248	248	248	248	248	248	248	248
	356	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
	357	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
	358	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247
	359	247	247	247	247	247	247	247	247	247	247	...	248	248	247	247	247	247	247	247	247	247

1080 rows × 640 columns

Array manipulation

Images are represented in Python with the type numpy.ndarray or "n-dimensional array." Grayscale images are 2-dimensional arrays with pixel luminance values indicated in each position. Color images are 3-dimensional arrays with pixel color values indicated for each channel (red, green, blue) in each position. Can you set the left and right sides of this picture to 0 so that Dubs II appears surrounded by black borders?

dubs = iio.imread("dubs.jpg")
dubs[:50, :25] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cdd4e650>

When we're performing an assignment on 2-dimensions of a 3-dimensional image, NumPy follows broadcasting rules to evaluate the operation. The simplest version of broadcasting are just element-wise operations.

# task: set the borders of left and right of the image to be black
dubs[:, :100] = 0
dubs[:, -100:] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cdd79de0>

dubs[:, :100].shape

(360, 100, 3)

# what if we just set the red channel?
dubs = iio.imread("dubs.jpg")
dubs[:, :100, 0] = 0
dubs[:, -100:, 0] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cdc20df0>

dubs[:, slice(0,100)] = 0

dubs = iio.imread("dubs.jpg")
dubs[:, :100] = dubs[:, -100:] = 0
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cddcd0f0>

Let's try a more complicated example. Using the floor division operator, fill in the imshow call to decrease only the green channel so that the overall picture is much more purple than before.

5 // 2

2

dubs = iio.imread("dubs.jpg")
plt.imshow(dubs // 2)

<matplotlib.image.AxesImage at 0x7ec7cda05ed0>

plt.imshow(dubs[1] // 2)

<matplotlib.image.AxesImage at 0x7ec7cddb3220>

dubs[:, :, 1] = dubs[:, :, 1] // 2 # decrease the green channel pixel value
plt.imshow(dubs)

<matplotlib.image.AxesImage at 0x7ec7cdb8a050>

plt.imshow(dubs // [1, 2, 1])

<matplotlib.image.AxesImage at 0x7ec7cda7a6b0>

plt.imshow(dubs // [:, 2, :])

  Cell In[45], line 1
    plt.imshow(dubs // [:, 2, :])
                        ^
SyntaxError: invalid syntax

Practice: Instafade

Write code to apply a fading filter to the image. The fading filter reduces all color values to 77% intensity and then adds 38.25 to each resulting color value. (These numbers are somewhat arbitrarily chosen to get the desired effect.)

The provided code converts the dog array from integer values to floating-point decimal values. To display the final image, the code converts the numbers in the dog array back to uint8 before passing the result to imshow.

dog = iio.imread("dog.jpg").astype("float32")
dog = dog * 0.77 + 38.25
plt.imshow(dog.astype("uint8"))

<matplotlib.image.AxesImage at 0x7ec7cd931930>

Practice: Gotham

Write code to apply the following operations to an image.

1. Expand the red colors by 50% by subtracting 128 from each red channel value, multiply the result by 1.5, and then add 128 to restore the original value range.
2. Increase the blue colors by 13 by adding 13 to each blue channel value.
3. Add black letterboxing bars by setting the top 150 and bottom 150 pixels to black.
4. Clip color values outside the range [0, 255] by reassign all values above 255 to 255 and all values below 0 to 0.

dog = iio.imread("dog.jpg").astype("float32")
dog[:, :, 0] = (dog[:, :, 0] - 128) * 1.5 + 128
dog[:, :, 2] += 13
dog[:150, :] = 0
dog[-150:, :] = 0
dog[dog > 255] = 255
dog[dog < 0] = 0
# np.clip()
plt.imshow(dog.astype("uint8"))

<matplotlib.image.AxesImage at 0x7ec7cd771000>

(dog > 255).shape # numpy equivalent of pandas boolean series

(1000, 1600, 3)

dog[dog > 255]

array([260. , 257. , 267.5, ..., 266. , 261.5, 257. ], dtype=float32)

if dog>255: # if boolean_expression
    pass

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[55], line 1
----> 1 if dog>255: # if boolean_expression
      2     pass

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Optional: Advanced broadcasting

What is the result of adding the following two arrays together following the broadcasting rules?

x = np.array([[1], [2], [3]])
x

array([[1],
       [2],
       [3]])

y = np.array([1, 2, 3])
y

array([1, 2, 3])

x.shape, y.shape

((3, 1), (3,))

x + y

array([[2, 3, 4],
       [3, 4, 5],
       [4, 5, 6]])

(3 x 1)
    (3)