Lec 15. Generative Models: Representation Learning Meets Generative Modeling

Quick Overview

Variational Autoencoders (VAEs) tightly couple generative modeling and representation learning by employing an autoencoder structure where the encoder maps data to a simple, sampleable prior distribution, typically a Gaussian, allowing the decoder to generate new data, effectively modeling the data distribution as an infinite mixture of Gaussians parameterized by a neural network.

Key Points: Generative modeling and representation learning are tightly coupled, and VAEs explicitly model both directions simultaneously: representation learning maps data to abstract representations (encoder f), and generative modeling maps representations to data (decoder g). A VAE is fundamentally an autoencoder where the encoder maps data not to an arbitrary latent space, but specifically to a simple, sampleable distribution, such as a Gaussian, ensuring that new samples can be generated by sampling from this prior and passing through the decoder. VAEs model the data distribution as an infinite mixture of Gaussians, parameterized by a neural network (g) that maps points from a simple prior distribution (pz, unit Gaussian) to the means and variances of the mixture components. The objective function for the generative model (p) involves maximizing the marginal likelihood, which requires calculating an intractable integral over the latent variables z, necessitating approximation techniques like Monte Carlo estimation. The core VAE 'trick' involves using importance sampling by introducing an encoder (f) that models the posterior distribution p(z|x) as a tractable distribution q(z|x), which is often a Gaussian parameterized by the encoder's output (mean and variance). The final objective function maximized is the Evidence Lower Bound (ELBO), which acts as a lower bound to the data log likelihood and comprises two terms: a reconstruction/likelihood term and a regularization term penalizing the difference between the inferred distribution q(z|x) and the prior p(z), typically measured by KL divergence.