Lec 17. Generalization: Out-of-Distribution (OOD)

Quick Overview

Out-of-distribution (OOD) generalization remains one of modern machine learning's biggest open challenges, as models trained under IID assumptions fail when encountering real-world changes like adversarial examples or distribution shifts, necessitating new evaluation and robustness methods.

Key Points: Simple categorization tasks like ImageNet classification are largely solved, but real-world deployment, like in self-driving cars, fails due to the inability to guarantee model robustness against unseen data. Deployed AI often functions as "high precision human verification systems," such as ChatGPT or medical decision support tools, where a human corrects errors, masking issues that arise when models are trusted without verification. Adversarial examples, created via projected gradient ascent optimizing for maximum loss against a target class, demonstrate model brittleness by changing predictions with imperceptible pixel perturbations. Adversarial training involves a nested optimization loop, finding the worst-case perturbation for every data point and training the model to be robust against it, which is computationally expensive. Distributionally Robust Optimization (DRO) addresses distribution shifts by assuming the worst-case shift occurs and optimizing the model to minimize loss across all possible perturbed distributions, effectively reweighting attention to minority subgroups. Many model errors stem from learning 'shortcuts'—spurious correlations in the training data, such as an X-ray image containing a ruler indicating a likely tumor, rather than the tumor itself. Realistic distribution shifts, exemplified by the Wilds benchmark, cause significant performance degradation (e.g., 16% macro F1 drop) due to complex factors like background changes, sensor updates, and subpopulation shifts.

Context: The lecture initiates a discussion on Out-of-Distribution (OOD) generalization, which addresses the critical machine learning problem of ensuring models perform reliably on data outside their training distribution. The speaker, Sara Beery, contrasts the success seen in controlled environments like ImageNet classification with the failures in high-stakes applications like autonomous driving, framing the challenge around robustness and risk.