Dr. Tiziana Ligorio x CSCI 493.77 - Deep Learning, Hunter College of the City University of New York

CNN’s main components include convolutional layers, pooling layers and dense layers.

• A convolutional layer applies filters (or kernels) to its input to extract features. It does so by performing a convolution (slides the kernel across the input to compute neurons in the output of the layer).
• A pooling layer reduces the dimension of its input feature maps.
• Dense layers are used to extract the final predictions.

Image credits: Gèron, Hands On ML3

Filters / Kernels

The terms filter and kernel are used interchangeably.

You can think of the filter as a 2D matrix that slides across the input of the convolution. Each neuron in the next layer is equivalent to summing the results of the element-wise multiplication of the patch and a same-size filter.

• Essentially, the filter(s) are the weights we aim to learn
• Weights are shared across patches. Patches overlap.
• This is an awesome website that illustrates image kernels (filters) in action. The idea precedes CNNs and you are probably familiar with this idea from applying filters in image editors. In CNN however, we don’t start with definite filters (e.g. the blur filter), instead we aim to learn the filters that help our model learn the features it needs tdiscern the image.

Color image & Multiple Filters

The output of applying a single filter is called a Feature Map. The output of applying multiple filters is a stack of feature maps and has the same spatial dimensions as the input. The number of feature maps produced (the depth of the output) is the same as the number of filters applied.

But, of course, if this network of convolutional layers is to predict non-linear functions, we need to add nonlinearity here too, just as we would in a network of dense layers.