4: Deep Learning for Computer Vision – Transfer Learning and Fine-Tuning; Intro to HuggingFace

Quick Overview

Convolutional Neural Networks (CNNs) address shortcomings of flattening images for dense layers by preserving spatial adjacency, reducing parameter count, and enabling translation invariance through the use of convolutional filters which learn features like lines and edges directly from the data via backpropagation, achieving 90.5% accuracy on Fashion MNIST in a demonstration.

Key Points: Flattening an image before feeding it to a dense layer causes the loss of spatial adjacency information, leads to too many parameters (e.g., 2.7 billion for a 3024x3024 color image to a 100-neuron layer), and ignores translation invariance (the ability to detect a feature regardless of its location). Convolutional layers utilize atomic building blocks called convolutional filters, which are small matrices of numbers designed to detect specific features like lines or curves; the breakthrough was learning these filter numbers from data using backprop rather than designing them by hand. The convolution operation involves superimposing a filter onto a section of the image, multiplying matching numbers, summing them, and running the result through a ReLU activation function to generate an output cell. When handling color images (3 channels), the 2D filter is made 3D, matching the input depth (e.g., 3x3x3), where all matching numbers are multiplied and summed to produce a single output value, which combines information across the RGB channels. Pooling layers (max pooling or average pooling) are used after convolutional layers to down-sample the tensor, reducing its size while forcing the network to summarize features; max pooling acts like an OR condition, checking if any feature fired strongly in a local region. A basic CNN architecture consists of sequential convolutional blocks (one or two Conv2D layers followed by a MaxPool2D layer), which progressively reduce height/width while increasing depth (number of filters), concluding with a flattening layer connected to dense layers and a softmax output. The instructor built a CNN for Fashion MNIST using Keras, achieving 90.5% accuracy after 10 epochs, significantly improving upon the 88% accuracy achieved by the previous flattened network approach.