Deployment & Responsible Vision

A vision model is only useful when it runs where it's needed — a phone, an edge camera at a junction, a hospital server, a cloud GPU — within that device's limits:

• Latency — time for one prediction (a safety system may need < 30 ms).
• Throughput — predictions per second (a video stream at 30 fps).
• Memory — RAM / VRAM for weights and activations.
• Power and cost — battery life, energy bills, cloud spend.

Latency and throughput are different: batching improves throughput but increases each request's latency. Always benchmark on the target device, with realistic inputs, after a warm-up, reporting p50 and p99.

• Quantisation — store weights (and often activations) as INT8 instead of FP32: 4× smaller and faster on hardware with integer acceleration. Post-training quantisation needs a small calibration set; quantisation-aware training recovers more accuracy.
• Pruning — remove weights or whole channels that contribute little; structured pruning gives real speed-ups.
• Knowledge distillation — train a small "student" to mimic a large "teacher's" outputs.
• Efficient architectures — MobileNet, EfficientNet, small YOLO variants.

Then export to an inference runtime: ONNX as the common exchange format; TensorRT for NVIDIA GPUs; OpenVINO for Intel CPUs; TFLite or Core ML for phones. Re-validate accuracy after every step.

Grad-CAM shows which image regions drove a CNN's prediction:

1. Run the image forward and pick the class score y_c.
2. Compute the gradient of y_c with respect to the last convolutional feature maps A^k.
3. Global-average-pool each gradient map to get a weight α_k per channel ("how important is this channel for class c").
4. Heatmap = ReLU(Σ_k α_k A^k), upsampled to the image size and overlaid.

ReLU keeps only regions that increase the class score. Grad-CAM is how you catch shortcut learning — e.g. a pneumonia model that actually looks at the hospital's text marker in the corner, or a "husky" detector that looks at snow.

A single overall accuracy can hide who the model fails. The Gender Shades study found commercial gender classifiers with error rates up to 34.7% for darker-skinned women versus 0.8% for lighter-skinned men.

Practices to know:

• Disaggregated evaluation — report metrics per subgroup (skin tone, age, region, camera type), and fix gaps with better data.
• Model cards — document intended use, out-of-scope uses, training data, per-group performance, limitations and ethical considerations.
• Robustness testing — small adversarial perturbations or physical patches/stickers can flip predictions; test against noise, blur, lighting and distribution shift.
• Privacy — faces and licence plates are personal data; minimise retention, blur where possible, and follow data-protection law (e.g. Uganda's Data Protection and Privacy Act 2019).