If we want to use [8, ‘pool’, 16, "pool"] as structure for our deep convolutional neural network, what is the shape of tensor for each step? Assume we use default parameters for convolutional layer (kernel size 3, stride 1) and pooling (kernel size = stride = 2) layer.
Layer | Output Size | Output Channels |
---|---|---|
Input | 30 x 30 | 3 |
Conv | 28 x 28 | 8 |
ReLU | 28 x 28 | 8 |
Pool | 14 x 14 | 8 |
Conv | 12 x 12 | 16 |
ReLU | 12 x 12 | 16 |
Pool | 6 x 6 | 16 |
Linear | 5 |
import torch
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt
import numpy as np
x = torch.randn(size=(16,3,30,30))
print(x.shape)
plt.imshow(np.transpose(x[0,:], (1,2,0)))
plt.show()
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
torch.Size([16, 3, 30, 30])
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.l1 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3)
self.pool = nn.MaxPool2d(2,2)
self.l2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3)
self.fc1 = nn.Linear(32 * 6 * 6, 5)
def forward(self, x):
x = F.relu(self.l1(x))
print(x.shape)
x = self.pool(x)
print(x.shape)
x = F.relu(self.l2(x))
print(x.shape)
x = self.pool(x)
print(x.shape)
# Note here!!!
# Important! You need to reshape the tensor to two dimensional (batch_size, num_features)
x = x.reshape(-1, 32*6*6)
x = self.fc1(x)
return x
m = CNN()
y = m(x)
print('Output size', y.shape)
torch.Size([16, 16, 28, 28]) torch.Size([16, 16, 14, 14]) torch.Size([16, 32, 12, 12]) torch.Size([16, 32, 6, 6]) Output size torch.Size([16, 5])
print("Number of Parameters (without bias) in First Conv Layer:", m.l1.weight.shape)
Number of Parameters (without bias) in First Conv Layer: torch.Size([16, 3, 3, 3])