# Lecture 23: Introduction to Machine Learning Source: https://www.youtube.com/watch?v=kTsieIl_YBA Recap page: https://rapidrecap.app/video/kTsieIl_YBA Generated: 2025-12-03T16:11:07.587+00:00 --- ## Quick Overview Machine learning, central to the Fourth Industrial Revolution, differs from traditional statistics by prioritizing data collection before hypothesis development, and it encompasses unsupervised learning for pattern detection, supervised learning for prediction using training/validation/test sets to avoid overfitting, and reinforcement learning for multi-stage decision-making, as demonstrated by John Hull's work applying neural networks to derivative pricing and reinforcement learning to hedging strategies. **Key Points:** - Machine learning marks the Fourth Industrial Revolution, contrasting with the Third Industrial Revolution (computers and digitization) by enabling systems to learn relationships from data rather than just automating known rules, such as determining loan approval rules instead of just automating existing ones. - Machine learning fundamentally differs from traditional statistics because practitioners collect data first to find patterns or develop predictive tools, whereas statistics requires developing a hypothesis before collecting data. - The three primary types of machine learning algorithms discussed are Unsupervised learning (looking for patterns/clusters), Supervised learning (predicting or classifying using training, validation, and test sets to manage overfitting), and Reinforcement learning (multi-stage decision making). - Artificial Neural Networks (ANNs) predict output through multiple stages (hidden layers) using weighted combinations of previous layer values passed through activation functions (like sigmoid or identity), with the weights being the primary parameters determined through iterative gradient descent. - In supervised learning, overfitting occurs when a complex model fits the training set too well but performs poorly on the validation set; the rule of thumb is to increase complexity until performance worsens on the validation set. - John Hull demonstrated using an ANN trained on 10,000 Black-Scholes-Merton prices (with added noise) to replicate the model quickly, showing real-world application in valuing exotic derivatives faster than Monte Carlo simulation. - Reinforcement learning, used for sequential decision-making, involves a trial-and-error process balancing exploration (random choice) and exploitation (choosing the best known action), successfully used to solve the game of Nim and applied to hedging derivatives to potentially save transaction costs. **Context:** John Hull, a long-time professor at the University of Toronto's Rotman School of Management known for his work on derivatives, introduced machine learning concepts to the class, noting he switched his primary focus to this area about seven or eight years prior due to its growing importance. His presentation covered the nature of machine learning, applications of neural networks, and applications of reinforcement learning, drawing parallels between historical industrial revolutions and the role of AI/ML within them. ## Detailed Analysis John Hull introduced machine learning as the core of the Fourth Industrial Revolution, distinct from the Third (digitization) because it allows computers to learn relationships from data rather than just automating predefined rules, exemplified by using ML to derive loan approval rules rather than just automating existing ones. He stressed the methodological difference from statistics: ML collects data first to find patterns, while statistics demands a prior hypothesis. Hull categorized algorithms into Unsupervised (pattern finding), Supervised (prediction via training/validation/test splits to prevent overfitting), and Reinforcement Learning (sequential decision making). Neural networks operate in stages, using weights and activation functions (like sigmoid, hyperbolic tangent, or ReLU) to relate inputs across hidden layers; model parameters (weights) are learned by minimizing a function using gradient calculations, often employing backpropagation, while carefully choosing a learning rate to avoid slow convergence or oscillation. Hull provided a financial application where an ANN successfully replicated Black-Scholes-Merton option prices, arguing this technique accelerates the valuation of slow-to-calculate exotic derivatives. Furthermore, he detailed research using ML to model the relationship between asset returns and implied volatility changes, finding that including the VIX index as a feature significantly improved results by capturing different volatility surface behaviors in high vs. low volatility environments. Finally, he discussed reinforcement learning's success in games like Nim, framing it as a sequence of decisions (actions, states, rewards) balancing exploration and exploitation, and noted its application in hedging derivatives where looking several periods ahead can save transaction costs compared to instantaneous Greek letter hedging, especially for complex options like barrier options. ### Introduction and Context - John Hull's background in math and operational research (shoe retailer) before moving to U of T for 30 years, focusing on derivatives then shifting to machine learning; He authored 'Machine Learning in Business' and is preparing the fourth edition. ### Machine Learning Fundamentals - ML is central to the Fourth Industrial Revolution (AI/ML); Techniques are old, but machine speed makes them practical; Key distinction from traditional automation is learning relationships from data rather than automating known rules. ### ML vs. Statistics - Machine learning collects data first to explain it and find patterns, contrasting with statistics where a hypothesis must be developed before data collection. ### Machine Learning Types - Algorithms include Unsupervised learning (finding clusters) - Supervised learning (predicting/classifying using 60% training, 20% validation, 20% test splits to check for overfitting) - Reinforcement learning (multi-stage decision making). ### Artificial Neural Networks (ANNs) Structure - ANNs process inputs through multiple hidden layers sequentially, unlike single-step models like linear regression; Parameters are the weights applied between layers, and activation functions (sigmoid, ReLU) introduce non-linearity; Gradient descent calculates partial derivatives (via backpropagation) to minimize an objective function, requiring careful tuning of the learning rate. ### ANN Application Example - Hull trained an ANN (3 hidden layers, 20 neurons/layer, sigmoid activation on hidden layers, identity on the final layer) on 10,000 generated Black-Scholes-Merton prices, demonstrating that ML can replicate complex pricing models quickly, useful for exotic derivatives valuation where Monte Carlo simulation is slow. ### Volatility Modeling Research - Research investigated the relationship between asset returns and implied volatility change, finding that including the VIX index as a feature yielded a 60% improvement over simpler models by recognizing different volatility surface behaviors in high/low volatility environments. ### Reinforcement Learning (RL) - RL focuses on finding a strategy for a series of decisions in an unpredictable environment, using actions, states, and rewards; It employs exploration (random choice) versus exploitation (best known choice), balancing these via a decaying epsilon parameter, as illustrated by solving the Nim game where the best strategy is to leave the opponent with 4n+1 matches.