# Adam Marblestone – AI is missing something fundamental about the brain Source: https://www.youtube.com/watch?v=_9V_Hbe-N1A Recap page: https://rapidrecap.app/video/_9V_Hbe-N1A Generated: 2025-12-30T18:01:42.24+00:00 --- ## Quick Overview The fundamental missing piece in current AI, particularly LLMs, compared to the human brain, likely lies in the highly specific and evolutionarily complex loss functions the brain utilizes, rather than just architecture or simple next-token prediction objectives used in machine learning. **Key Points:** - The speaker posits that current LLMs, despite massive data, possess only a small fraction of human capabilities, suggesting a fundamental element is missing in AI design. - The speaker's hunch is that the field has neglected the role of very specific loss functions, contrasting them with the simple, mathematically simple ones like cross-entropy used in ML. - A possibility for the cortex is that it functions as an 'incredibly general prediction engine' capable of 'omnidirectional inference,' predicting any subset of variables from any other subset, unlike LLMs which natively predict only the next token. - Steve Byrnes' theory suggests the brain separates into a 'Learning Subsystem' (cortex) and a 'Steering Subsystem' (innate responses/rewards), where the cortex learns to predict the Steering Subsystem's outputs. - The genomic evidence suggests far more diverse and specialized cell types exist in the Steering Subsystem compared to the more repeating, general architecture of the Learning Subsystem. - LLMs use a 'dumb' form of RL conceptually, lacking value functions, whereas parts of the brain like the striatum and basal ganglia likely perform model-free RL, while the cortex engages in model-based reasoning. **Context:** The discussion centers on addressing the 'million-dollar question': why current AI models, like Large Language Models (LLMs), significantly lag behind human capabilities despite being trained on vastly more data. The speaker explores this gap by comparing current deep learning paradigms (architecture, learning algorithms, cost functions) against the known and hypothesized structures of the human brain, drawing heavily on concepts synthesized by neuroscientist Steve Byrnes, who proposes a dual system involving a general Learning Subsystem and an innate Steering Subsystem. ## Detailed Analysis Adam Marblestone argues that cracking the code of brain intelligence requires empowering neuroscience technologically because current AI lacks fundamental components present in the brain. He focuses on the learning framework, suggesting that machine learning oversimplifies cost functions (e.g., cross-entropy) while evolution likely encoded complex, curriculum-generating loss functions. He explores the concept of the cortex as an 'omnidirectional inference' engine, capable of predicting any pattern from any other subset of inputs, contrasting this with the unidirectional next-token prediction of LLMs, noting similarities to Yann LeCun's energy-based models. The analysis heavily incorporates Steve Byrnes' framework, dividing the brain into a Learning Subsystem (cortex) and a Steering Subsystem (innate drives/rewards). The Steering Subsystem possesses innate heuristics and primitive sensory systems (like the superior colliculus), and the cortex learns to predict these innate responses (e.g., predicting the 'spider reflex' based on the abstract word 'spider'). This framework explains how evolution builds complex desires without knowing future learned variables, by having the genome specify innate reward heuristics, which the cortex then wires its generalized learning onto. Furthermore, the discussion touches on RL, noting LLMs use a simple form of RL lacking value functions, unlike the basal ganglia which likely performs model-free RL, while the cortex handles model-based planning. Genomic evidence supports this dichotomy: far more diverse cell types exist in the Steering Subsystem, suggesting bespoke, evolutionarily programmed circuits for specific innate functions, while the cortex architecture is more general and repetitive, requiring less genomic specification. ### Missing AI Components - The brain's fundamental difference likely lies in neglected, specific loss functions encoded by evolution, contrasting with simple ML objectives like cross-entropy - The cortex might perform omnidirectional inference, predicting any variable subset from any other, unlike LLMs' next-token focus. ### The Learning vs. Steering Subsystems (Byrnes) - The brain separates into a Learning Subsystem (cortex) with general architecture and a Steering Subsystem responsible for innate rewards and reflexes, which the cortex learns to predict - Genomic evidence shows specialized cell types dominate the Steering Subsystem, supporting the idea of pre-wired innate functions. ### Reinforcement Learning in Brain vs. AI - LLMs employ a conceptually 'dumb' form of RL lacking value functions, whereas the basal ganglia likely performs model-free RL (like Atari Q-learning) - The cortex likely implements model-based RL, using its world model to predict future rewards generated by the Steering Subsystem. ### Hardware Tradeoffs - Biological hardware forces energy efficiency (low voltage, slow speed), but offers advantages like co-locating memory and compute, enabling 'cognitive dexterity' - Digital minds benefit from external read/write access and easier amortization of inference, but biological hardware might be inherently better suited for stochastic sampling methods.