# Lec 17. Generalization: Out-of-Distribution (OOD) Source: https://www.youtube.com/watch?v=tjD9LIzIIek Recap page: https://rapidrecap.app/video/tjD9LIzIIek Generated: 2026-02-11T15:05:56.081+00:00 --- ## Quick Overview Out-of-distribution (OOD) generalization remains one of modern machine learning's biggest open challenges, as models trained under IID assumptions fail when encountering real-world changes like adversarial examples or distribution shifts, necessitating new evaluation and robustness methods. **Key Points:** - Simple categorization tasks like ImageNet classification are largely solved, but real-world deployment, like in self-driving cars, fails due to the inability to guarantee model robustness against unseen data. - Deployed AI often functions as "high precision human verification systems," such as ChatGPT or medical decision support tools, where a human corrects errors, masking issues that arise when models are trusted without verification. - Adversarial examples, created via projected gradient ascent optimizing for maximum loss against a target class, demonstrate model brittleness by changing predictions with imperceptible pixel perturbations. - Adversarial training involves a nested optimization loop, finding the worst-case perturbation for every data point and training the model to be robust against it, which is computationally expensive. - Distributionally Robust Optimization (DRO) addresses distribution shifts by assuming the worst-case shift occurs and optimizing the model to minimize loss across all possible perturbed distributions, effectively reweighting attention to minority subgroups. - Many model errors stem from learning 'shortcuts'—spurious correlations in the training data, such as an X-ray image containing a ruler indicating a likely tumor, rather than the tumor itself. - Realistic distribution shifts, exemplified by the Wilds benchmark, cause significant performance degradation (e.g., 16% macro F1 drop) due to complex factors like background changes, sensor updates, and subpopulation shifts. **Context:** The lecture initiates a discussion on Out-of-Distribution (OOD) generalization, which addresses the critical machine learning problem of ensuring models perform reliably on data outside their training distribution. The speaker, Sara Beery, contrasts the success seen in controlled environments like ImageNet classification with the failures in high-stakes applications like autonomous driving, framing the challenge around robustness and risk. ## Detailed Analysis The lecture establishes that while many supervised learning tasks are solved within their training distribution (IID), real-world deployment exposes fragility, especially in safety-critical domains like self-driving cars where robustness guarantees are lacking. Current deployed AI often relies on human verification loops, as seen with ChatGPT or healthcare decision support tools, which mitigates immediate failure but masks underlying OOD issues. The speaker details two main failure modes: adversarial examples and distribution shifts. Adversarial examples are created by optimizing small, targeted perturbations (often using projected gradient ascent) to maximize model loss or force a specific misclassification, revealing that models rely on brittle, non-robust features. Robustness can be sought through adversarial training, a computationally expensive nested optimization that forces the model to be good even in worst-case perturbations, leading to more human-intuitive feature learning. Distribution shifts involve larger changes between training and testing distributions, such as seasonal changes or sensor updates, as highlighted by the Wilds benchmark, which shows significant performance drops on realistic shifts. These shifts often reveal that models learn 'shortcuts'—spurious correlations in the training data, like associating bow ties with cats or hospital acquisition source with disease prediction. Distributionally Robust Optimization (DRO) offers a solution by optimizing over potential distribution perturbations, which is particularly effective in mitigating class imbalance and bias by forcing attention onto minority subgroups. ### Motivation for OOD - Success in ImageNet classification contrasts with failures in self-driving cars due to lack of robustness guarantees - Deployed AI often relies on human verification loops like ChatGPT. ### Adversarial Examples Explained - Small, optimized noise added to an input can drastically change model output, demonstrated by convincing models a turtle 3D print is a rifle - Creation involves projected gradient ascent optimizing for maximum change within an infinity norm constraint. ### Defending Against Perturbations - Adversarial training uses nested optimization to minimize loss over worst-case perturbations, acting as a targeted regularization or domain adaptation technique. ### The Role of Shortcuts - Models learn features correlated with the label in training data that are not robust to distribution shifts, such as relying on the presence of a ruler in an X-ray to diagnose malignancy. ### Distribution Shift Reality - Realistic shifts (Wilds benchmark) cause significant performance degradation (16% macro F1 drop) due to complex factors like background, pose, and subpopulation variation. ### Distributionally Robust Optimization (DRO) - This technique optimizes for the worst-case data distribution shift, effectively reweighting data to pay more attention to minority classes and mitigating bias concerns.