Images as Data

To a computer an image is a grid of numbers. A colour image of height H and width W has three channels (red, green, blue), so it's a 3-D array of shape H × W × 3, usually with 8-bit integers from 0 (dark) to 255 (bright). A 224 × 224 RGB image holds 224 × 224 × 3 = 150,528 numbers.

Conventions matter in code:

• NumPy, OpenCV and PIL use HWC (height, width, channels). PyTorch uses CHW, and a batch is NCHW.
• OpenCV loads images as BGR, not RGB — a classic source of "the model works in training but not on camera images".
• transforms.ToTensor() converts HWC uint8 [0, 255] into CHW float [0, 1].

RGB channels are strongly correlated: a shadow lowers all three together, so "brightness" is smeared across every channel. Other colour spaces separate it out:

• HSV — Hue (which colour), Saturation (how vivid), Value (how bright). Thresholding on hue finds "red objects" regardless of lighting far better than RGB thresholds.
• Grayscale — Y = 0.299R + 0.587G + 0.114B. The weights follow human sensitivity (we're most sensitive to green). One channel instead of three: less compute, but colour information is gone.
• LAB — perceptually uniform; distances roughly match how different colours look.

Deep networks usually take RGB and learn whatever colour features they need, but colour spaces still matter for classical pipelines and augmentation.

Networks train best when inputs are centred near zero with similar scales. The standard recipe:

1. Scale to [0, 1]: x = pixel / 255.
2. Standardise per channel: x̂ = (x − μ_c) / σ_c.

When fine-tuning an ImageNet-pretrained model, use ImageNet's statistics: μ = (0.485, 0.456, 0.406), σ = (0.229, 0.224, 0.225). The pretrained filters and BatchNorm layers were tuned to inputs distributed that way; feeding differently scaled data shifts every activation and wastes the pretraining.

tf = transforms.Compose([transforms.ToTensor(),
                         transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

Normalised values are z-scores: a pure white red channel becomes (1 − 0.485)/0.229 ≈ 2.25, and black becomes about −2.12.

A classic filter is a small grid of weights slid over the image (convolution). The Sobel-x kernel [[-1,0,1],[-2,0,2],[-1,0,1]] subtracts left neighbours from right ones, so it responds strongly where intensity changes horizontally — vertical edges. A box blur averages; a sharpen kernel boosts the centre against its neighbours. CNNs learn kernels like these.

Data augmentation creates new training views of each image: random crops and flips, small rotations, colour jitter, blur. It acts as a regulariser and teaches invariances — a flipped cat is still a cat. Augment only the training set, and only with transforms that keep the label true (flipping a "left turn" road sign changes its meaning!).

Not all vision sensors produce frames: event cameras report per-pixel brightness changes asynchronously with microsecond timing — no motion blur and enormous dynamic range, useful for fast robotics.