One layer of a convolutional network

Here is how we implement one layer of a convolutional network :

• We perform convolution of volume on the input image.
• We use number of filters depending on the number of features we want to detect
• To each of the resultant image we add an bias which is a real number and then apply a non-linear operation, such as ReLU on it.
• We then stack all the resultant images.

If you have read Convolution over volume, which you should, we will you the same example which we used there. The convolution operation used to give us resultant images of dimensions 4 x 4. We then do the following :

$$\text{ReLU}(\text{Resultant Image from convolution with a single kernel}+b_1)$$

This is done for all the resultant images from all kernels and then they are stacked onto one another. This are the operations of one layer of convolutional network.

Here if we consider a single filter then we have 28 parameters. 27 values of the filter and the bias.

So even if we apply ten filter we will get only about 280 parameters. So we can use this filters on images of large size, while having comparatively less number of parameters.

Notation

If l is a convolutional layer :

• $f^{[l]}$ - filter size of layer l
• $p^{[l]}$ - padding
• $s^{[l]}$ - stride
• Input dimension - $[n_H^{[l-1]},n_W^{[l-1]},n_C^{[l-1]}]$
• Output dimension - $[n_H^{[l]},n_W^{[l]},n_C^{[l]}]$ where $n^{[l]}=[\frac{n^{[l-1]}+2p^{[l]}-f^{[l]}}{s^{[l]}}+1]$ and $n_C^{[l]}$ is the number of filters we have used
• Filter dimension - $[f^{[l]},f^{[l]},n_C^{[l-1]}]$
• Activation - $a^{[l]}$ has same dimensions as the output. For vectorization approach if we have m examples then dimensions are $[m,n_H^{[l]},n_W^{[l]},n_C^{[l]}]$
• Weights dimension - $[f^{[l]},f^{[l]},n_C^{[l-1]},n_C^{[l]}]$
• Bias dimension - $[1,1,1,n_C[l]]$

Let us move onto to a Convolutional network which has more layers and see how it works.