9: Generative AI – Large Language Models (LLMs) and Retrieval Augmented Generation (RAG)

Quick Overview

The lecture contrasts BERT's masking self-supervised learning technique with next word prediction, which powers autoregressive models like GPT, requiring a crucial architectural modification called causal self-attention to prevent the model from 'peeking into the future' during training.

Key Points: Next word prediction is a form of self-supervised learning where the model predicts the last word of a fragment, or sequentially predicts the next word based only on preceding text. Using the standard Transformer encoder (like BERT) for next word prediction is problematic because self-attention allows the last word to see all preceding words, including the word it is supposed to predict, which is described as 'cheating' or 'peeking into the future'. The solution for next word prediction models is implementing 'causal self-attention' (or masked self-attention), which zeros out attention weights for future words, ensuring context only flows from past to present tokens. The loss function for next word prediction models remains cross-entropy, calculated as the sum or average of cross-entropies for every word position being predicted in the sequence. Generating text involves decoding—choosing a token from the vocabulary probability distribution (softmax output)—using strategies like greedy decoding (picking the highest probability) for determinism or random sampling for creativity. LLMs can easily 'go off the rails' if a mistake is sampled from the long tail of low-probability words, as the model cannot easily recover from such errors in subsequent predictions. Advanced decoding strategies like top-k sampling (selecting only the top K most likely words) and top-p or nucleus sampling (selecting words whose cumulative probability exceeds a threshold P) help focus sampling on the 'head' of the distribution.

Context: This lecture segment reviews the self-supervised learning technique of masking used in models like BERT and introduces a different, more powerful technique: next word prediction, which is fundamental to autoregressive language models such as the GPT family. The discussion centers on adapting the Transformer architecture to correctly implement next word prediction by preventing the model from accessing future information during sequence generation training.