NumPy¶

In this lesson, we'll learn about ways to represent and manipulate images in Python. By the end of this lesson, students will be able to:

  • Apply ndarray arithmetic and logical operators with numbers and other arrays.
  • Analyze the shape of an ndarray and index into a multidimensional array.
  • Apply arithmetic operators, indexing, and slicing to manipulate RGB images.

We'll need two new modules: imageio, which provides utilities to read and write images in Python, and numpy, which provides the data structures for representing images in Python.

In [8]:
import imageio.v3 as iio
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

Reading an image¶

Let's use imageio's imread method to load a color picture as a grid of pixels. To then show an image, we can plot its pixels using the matplotlib function imshow.

In [9]:
jax = iio.imread("jax.jpg")
plt.imshow(jax)
Out[9]:
<matplotlib.image.AxesImage at 0x793fc2af7fd0>
No description has been provided for this image

We could also read this image in as a black-and-white image using the mode="L" option to imread and cmap="gray" to imshow. "L" stands for "luminance," a method to convert color to grayscale. "cmap" stands for "colormap," saying to treat pixels as grayscale.

In [10]:
jax = iio.imread("jax.jpg", mode="L")
plt.imshow(jax, cmap="gray")
Out[10]:
<matplotlib.image.AxesImage at 0x793fd0f0f0d0>
No description has been provided for this image

What kind of Python object does imread return?

In [11]:
type(jax)
Out[11]:
numpy.ndarray
In [14]:
jax.shape
Out[14]:
(600, 600)
In [15]:
jax
Out[15]:
array([[109, 115, 115, ...,  48,  47,  45],
       [114, 116, 114, ...,  47,  47,  48],
       [122, 124, 122, ...,  45,  45,  47],
       ...,
       [116,  99,  94, ...,  79,  80,  83],
       [106,  93,  89, ...,  72,  81,  86],
       [105, 102, 100, ...,  76,  76,  94]], dtype=uint8)

Pandas uses NumPy to represent a Series of values, so many element-wise operations should seem familiar. In fact, we can load an image into a Pandas DataFrame and see that this grayscale image is really a 2-d array of grayscale values ranging from [0, 255].

In [17]:
pd.DataFrame(jax)
Out[17]:
0 1 2 3 4 5 6 7 8 9 ... 590 591 592 593 594 595 596 597 598 599
0 109 115 115 116 122 121 117 120 128 121 ... 49 49 46 47 47 48 48 48 47 45
1 114 116 114 113 118 119 116 117 131 124 ... 50 49 49 47 47 48 48 47 47 48
2 122 124 122 118 118 121 116 107 120 122 ... 50 50 50 47 45 47 47 45 45 47
3 123 123 124 121 115 115 116 113 118 127 ... 50 49 48 47 46 47 47 46 45 46
4 124 115 113 118 120 120 122 125 126 127 ... 49 49 47 48 48 47 47 48 48 47
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
595 128 125 128 124 112 106 109 118 124 123 ... 78 98 104 101 97 105 100 85 91 106
596 123 113 113 116 116 106 95 101 111 114 ... 70 79 85 93 109 111 100 86 83 97
597 116 99 94 108 125 119 103 103 121 124 ... 69 78 84 82 82 84 83 79 80 83
598 106 93 89 104 126 133 125 123 123 125 ... 62 69 82 81 57 55 68 72 81 86
599 105 102 100 105 119 133 137 132 116 113 ... 65 61 88 113 113 96 86 76 76 94

600 rows × 600 columns

What do you think the colorful DataFrame should look like?

In [18]:
jax_color = iio.imread("jax.jpg")
pd.DataFrame(jax_color)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[18], line 2
      1 jax_color = iio.imread("jax.jpg")
----> 2 pd.DataFrame(jax_color)

File /opt/conda/lib/python3.11/site-packages/pandas/core/frame.py:827, in DataFrame.__init__(self, data, index, columns, dtype, copy)
    816         mgr = dict_to_mgr(
    817             # error: Item "ndarray" of "Union[ndarray, Series, Index]" has no
    818             # attribute "name"
   (...)
    824             copy=_copy,
    825         )
    826     else:
--> 827         mgr = ndarray_to_mgr(
    828             data,
    829             index,
    830             columns,
    831             dtype=dtype,
    832             copy=copy,
    833             typ=manager,
    834         )
    836 # For data is list-like, or Iterable (will consume into list)
    837 elif is_list_like(data):

File /opt/conda/lib/python3.11/site-packages/pandas/core/internals/construction.py:314, in ndarray_to_mgr(values, index, columns, dtype, copy, typ)
    308     _copy = (
    309         copy_on_sanitize
    310         if (dtype is None or astype_is_view(values.dtype, dtype))
    311         else False
    312     )
    313     values = np.array(values, copy=_copy)
--> 314     values = _ensure_2d(values)
    316 else:
    317     # by definition an array here
    318     # the dtypes will be coerced to a single dtype
    319     values = _prep_ndarraylike(values, copy=copy_on_sanitize)

File /opt/conda/lib/python3.11/site-packages/pandas/core/internals/construction.py:592, in _ensure_2d(values)
    590     values = values.reshape((values.shape[0], 1))
    591 elif values.ndim != 2:
--> 592     raise ValueError(f"Must pass 2-d input. shape={values.shape}")
    593 return values

ValueError: Must pass 2-d input. shape=(600, 600, 3)
In [20]:
jax_color.shape
Out[20]:
(600, 600, 3)
In [21]:
jax.shape
Out[21]:
(600, 600)

Array manipulation¶

Images are represented in Python with the type numpy.ndarray or "n-dimensional array." Grayscale images are 2-dimensional arrays with pixel luminance values indicated in each position. Color images are 3-dimensional arrays with pixel color values indicated for each channel (red, green, blue) in each position. Can you set the left and right sides of this picture to 0 so that Jax appears surrounded by black borders?

In [43]:
jax = iio.imread("jax.jpg")
jax[:100, :] = 0
jax[-100:, :] = 0
plt.imshow(jax)
Out[43]:
<matplotlib.image.AxesImage at 0x793fc1b86e90>
No description has been provided for this image
In [46]:
jax[:50, :25] = [0, 0, 1]
jax[:50, :25]
Out[46]:
array([[[0, 0, 1],
        [0, 0, 1],
        [0, 0, 1],
        ...,
        [0, 0, 1],
        [0, 0, 1],
        [0, 0, 1]],

       [[0, 0, 1],
        [0, 0, 1],
        [0, 0, 1],
        ...,
        [0, 0, 1],
        [0, 0, 1],
        [0, 0, 1]],

       [[0, 0, 1],
        [0, 0, 1],
        [0, 0, 1],
        ...,
        [0, 0, 1],
        [0, 0, 1],
        [0, 0, 1]],

       ...,

       [[0, 0, 1],
        [0, 0, 1],
        [0, 0, 1],
        ...,
        [0, 0, 1],
        [0, 0, 1],
        [0, 0, 1]],

       [[0, 0, 1],
        [0, 0, 1],
        [0, 0, 1],
        ...,
        [0, 0, 1],
        [0, 0, 1],
        [0, 0, 1]],

       [[0, 0, 1],
        [0, 0, 1],
        [0, 0, 1],
        ...,
        [0, 0, 1],
        [0, 0, 1],
        [0, 0, 1]]], dtype=uint8)
In [35]:
jax.shape
Out[35]:
(600, 600, 3)
In [34]:
nums = [1, 2, 3, 4]
nums[:2] = [0, 0]
nums
Out[34]:
[0, 0, 3, 4]

When we're performing an assignment on 2-dimensions of a 3-dimensional image, NumPy follows broadcasting rules to evaluate the operation. The simplest version of broadcasting are just element-wise operations.

In [68]:
plt.imshow(jax + 70)
Out[68]:
<matplotlib.image.AxesImage at 0x793fc1603010>
No description has been provided for this image

Let's try a more complicated example. Using the floor division operator, fill in the imshow call to decrease only the green channel so that the overall picture is much more purple than before.

In [57]:
3 // 2
Out[57]:
1
In [58]:
type(jax)
Out[58]:
numpy.ndarray
In [62]:
jax[:, :, 1] // 2
Out[62]:
array([[59, 62, 62, ..., 24, 23, 22],
       [61, 62, 61, ..., 23, 23, 24],
       [65, 66, 65, ..., 22, 22, 23],
       ...,
       [64, 55, 52, ..., 44, 44, 46],
       [59, 52, 50, ..., 40, 45, 48],
       [58, 57, 55, ..., 42, 43, 52]], dtype=uint8)
In [ ]:
jax[:, :, 1] // 2
In [63]:
jax // 2
Out[63]:
array([[[43, 59, 60],
        [46, 62, 63],
        [46, 62, 63],
        ...,
        [25, 24, 21],
        [25, 23, 21],
        [24, 22, 20]],

       [[46, 61, 62],
        [47, 62, 63],
        [46, 61, 62],
        ...,
        [25, 23, 21],
        [25, 23, 21],
        [25, 24, 21]],

       [[50, 65, 66],
        [51, 66, 67],
        [50, 65, 66],
        ...,
        [24, 22, 20],
        [24, 22, 20],
        [25, 23, 21]],

       ...,

       [[43, 64, 67],
        [35, 55, 58],
        [33, 52, 56],
        ...,
        [27, 44, 48],
        [28, 44, 49],
        [29, 46, 50]],

       [[37, 59, 62],
        [32, 52, 55],
        [30, 50, 53],
        ...,
        [23, 40, 45],
        [28, 45, 49],
        [29, 48, 52]],

       [[37, 58, 62],
        [35, 57, 60],
        [36, 55, 59],
        ...,
        [25, 42, 47],
        [24, 43, 47],
        [33, 52, 56]]], dtype=uint8)
In [64]:
jax // [1, 2, 1]
Out[64]:
array([[[ 87,  59, 120],
        [ 93,  62, 126],
        [ 93,  62, 126],
        ...,
        [ 51,  24,  43],
        [ 50,  23,  42],
        [ 48,  22,  40]],

       [[ 92,  61, 125],
        [ 94,  62, 127],
        [ 92,  61, 125],
        ...,
        [ 50,  23,  42],
        [ 50,  23,  42],
        [ 51,  24,  43]],

       [[100,  65, 133],
        [102,  66, 135],
        [100,  65, 133],
        ...,
        [ 48,  22,  40],
        [ 48,  22,  40],
        [ 50,  23,  42]],

       ...,

       [[ 87,  64, 134],
        [ 70,  55, 117],
        [ 66,  52, 112],
        ...,
        [ 55,  44,  97],
        [ 56,  44,  98],
        [ 58,  46, 101]],

       [[ 75,  59, 125],
        [ 64,  52, 111],
        [ 61,  50, 107],
        ...,
        [ 47,  40,  90],
        [ 56,  45,  99],
        [ 59,  48, 104]],

       [[ 74,  58, 124],
        [ 71,  57, 121],
        [ 72,  55, 118],
        ...,
        [ 51,  42,  94],
        [ 49,  43,  94],
        [ 67,  52, 112]]])
In [65]:
jax = iio.imread("jax.jpg")
plt.imshow(jax // [1, 2, 1])
Out[65]:
<matplotlib.image.AxesImage at 0x793fc1667790>
No description has been provided for this image

Practice: Instafade¶

Write code to apply a fading filter to the image. The fading filter reduces all color values to 77% intensity and then adds 38.25 to each resulting color value. (These numbers are somewhat arbitrarily chosen to get the desired effect.)

The provided code converts the oj array from integer values to floating-point decimal values. To display the final image, the code converts the numbers in the oj array back to uint8 before passing the result to imshow.

In [74]:
oj = iio.imread("oj.jpg").astype("float32")
oj *= 0.77
oj += 38.25
plt.imshow(oj.astype("uint8"))
Out[74]:
<matplotlib.image.AxesImage at 0x793fc14f9710>
No description has been provided for this image

Practice: Image Color Manipulation¶

Write code to apply the following operations to an image.

  1. Expand the red colors by 50% by subtracting 128 from each red channel value, multiply the result by 1.5, and then add 128 to restore the original value range.
  2. Increase the blue colors by 25 by adding 25 to each blue channel value.
  3. Add black letterboxing bars by setting the top 150 and bottom 150 pixels to black.
  4. Clip color values outside the range [0, 255] by reassign all values above 255 to 255 and all values below 0 to 0.
In [77]:
np.clip([1, 2, 3, 4, 5,6 , 6, 7, 8], a_min=0, a_max=4)
Out[77]:
array([1, 2, 3, 4, 4, 4, 4, 4, 4])
In [81]:
oj = iio.imread("oj.jpg").astype("float32")
oj *= [0.5, 1, 1]
oj += [0, 0, 25]
oj[:100, :] = 0
oj[-100:, :] = 0
oj = np.clip(oj, a_min=0, a_max=255)
plt.imshow(oj.astype("uint8"))
Out[81]:
<matplotlib.image.AxesImage at 0x793fc12511d0>
No description has been provided for this image

Optional: Advanced broadcasting¶

What is the result of adding the following two arrays together following the broadcasting rules?

In [ ]:
x = np.array([[1], [2], [3]])
x
In [ ]:
y = np.array([1, 2, 3])
y