3: Deep Learning for Computer Vision – Building Convolutional Neural Networks from Scratch

Quick Overview

The lecture focuses on building a Keras model for a heart disease problem and then transitions into explaining the fundamentals of training neural networks using Stochastic Gradient Descent (SGD) with minibatches, defining epochs, and introducing the concepts of overfitting, regularization methods like early stopping, and the role of Tensors in TensorFlow and Keras.

Key Points: The overall neural network training flow involves defining a network, calculating predictions, using a loss function to determine discrepancy, sending this to an optimizer to calculate gradients, and updating weights, a process that repeats. Stochastic Gradient Descent (SGD) processes data in small, randomly chosen subsets called minibatches, allowing for weight updates after each batch, unlike standard Gradient Descent which updates only once per epoch after processing all data. An epoch is defined as making one full pass through all the training data, and in SGD, the number of weight updates within an epoch equals the number of batches, calculated by dividing the training set size by the batch size and rounding up. Overfitting occurs when a model becomes too complex, fitting the idiosyncrasies of the training data, causing training error to decrease while validation error starts to climb; regularization methods like early stopping address this. Early stopping works by splitting data into training and validation sets and stopping training when the validation set error starts to climb, a technique Geoff Hinton called a "beautiful free lunch." Tensors are generalizations of numbers (rank 0), vectors (rank 1), and tables (rank 2) to higher dimensions; for instance, a color image is a rank 3 tensor, and a video stream of color images is a rank 4 tensor. TensorFlow automatically calculates gradients and handles parallel computation distribution across servers and GPUs, while Keras sits on top, providing convenience layers, activation functions, and easy training methods, with the functional API being the primary focus for this course.