Quiz

If we want to use [8, ‘pool’, 16, "pool"] as structure for our deep convolutional neural network, what is the shape of tensor for each step? Assume we use default parameters for convolutional layer (kernel size 3, stride 1) and pooling (kernel size = stride = 2) layer.

Layer Output Size Output Channels
Input 30 x 30 3
Conv 28 x 28 8
ReLU 28 x 28 8
Pool 14 x 14 8
Conv 12 x 12 16
ReLU 12 x 12 16
Pool 6 x 6 16
Linear 5
In [1]:
import torch
import torch.nn as nn
import torch.nn.functional as F

import matplotlib.pyplot as plt
import numpy as np
In [2]:
x = torch.randn(size=(16,3,30,30))
print(x.shape)


plt.imshow(np.transpose(x[0,:], (1,2,0)))
plt.show()
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
torch.Size([16, 3, 30, 30])
In [7]:
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        
        self.l1 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3)
        self.pool = nn.MaxPool2d(2,2)
        
        self.l2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3)

        self.fc1 = nn.Linear(32 * 6 * 6, 5)
        
    def forward(self, x):
        x = F.relu(self.l1(x))
        print(x.shape)
        x = self.pool(x)
        print(x.shape)
        x = F.relu(self.l2(x))
        print(x.shape)
        x = self.pool(x)
        print(x.shape)

        # Note here!!! 
        # Important! You need to reshape the tensor to two dimensional (batch_size, num_features)
        x = x.reshape(-1, 32*6*6) 
        
        x = self.fc1(x)
        return x
In [8]:
m = CNN()
y = m(x)
print('Output size', y.shape)
torch.Size([16, 16, 28, 28])
torch.Size([16, 16, 14, 14])
torch.Size([16, 32, 12, 12])
torch.Size([16, 32, 6, 6])
Output size torch.Size([16, 5])
In [14]:
print("Number of Parameters (without bias) in First Conv Layer:", m.l1.weight.shape)
Number of Parameters (without bias) in First Conv Layer: torch.Size([16, 3, 3, 3])