Convolutional Neural Networks

Flatten a 224 × 224 × 3 image and you have 150,528 inputs. A single dense layer to 1,000 neurons needs 150 million weights — slow, memory-hungry and a recipe for overfitting. Worse, it treats a pixel in the top-left corner as unrelated to its neighbour, and has to learn "cat ear" separately at every position.

Convolutional networks build in two assumptions (inductive biases) about images:

• Local connectivity — each neuron looks at a small patch (its receptive field), because nearby pixels are related.
• Parameter sharing — the same small kernel slides across the whole image, because a useful feature (an edge, an eye) is useful everywhere.

Output size: O = ⌊(W − K + 2P) / S⌋ + 1 for input width W, kernel K, padding P, stride S.

• "Same" convolution: K = 3, P = 1, S = 1 keeps the size (32 → 32).
• No padding shrinks by K − 1 (32 → 30 with K = 3).
• Stride 2 roughly halves it (224 → 112 with K = 3, P = 1).

Parameters: each of C_out filters has K × K × C_in weights plus one bias: (K·K·C_in + 1) · C_out. A 3 × 3 conv from 64 to 128 channels has (9·64 + 1)·128 = 73,856 parameters — independent of the image size, thanks to sharing.

Compute (multiply-adds): O × O × K × K × C_in × C_out — this does scale with image size, which is why resolution drives inference cost.

• Equivariance — shift the input and the output shifts the same way. Convolution is translation-equivariant: if the cat moves 10 px right, its feature-map activation moves right too.
• Invariance — shift the input and the output doesn't change. A classifier's final "cat" score should be (approximately) invariant.

Max pooling (e.g. 2 × 2, stride 2) keeps the largest value in each window. It halves height and width, has zero learnable parameters, and adds a little invariance to small shifts. Stacking conv + pooling, plus global average pooling at the end, moves the network from equivariant feature maps to an invariant prediction.

Each layer sees a larger region of the original image — its receptive field grows with depth (two stacked 3 × 3 convs see 5 × 5; three see 7 × 7, with fewer parameters than one 7 × 7). So features build up:

• early layers: edges, colours, simple textures;
• middle layers: patterns, corners, motifs;
• deep layers: object parts (eyes, wheels) and whole objects.

That's why early layers transfer well between tasks (module 4) and why visualising feature maps is a useful debugging tool. Modern CNNs (ResNets) add skip connections so very deep stacks still train, and batch normalisation to stabilise activations.