# Advice for beginners in AI: How to learn and what to build | Lex Fridman Podcast Source: https://www.youtube.com/watch?v=dGqhTpsu_5Y Recap page: https://rapidrecap.app/video/dGqhTpsu_5Y Generated: 2026-02-05T13:04:16.603+00:00 --- ## Quick Overview The main advice for beginners interested in AI, particularly those looking to build models from scratch or work in research, is to focus on understanding the fundamental mathematical and engineering concepts underlying large language models (LLMs), rather than immediately jumping into complex, high-level techniques like RLHF, to avoid burnout and ensure a solid foundation for long-term career growth. **Key Points:** - Beginners interested in AI should prioritize building foundational knowledge of LLM engineering and mathematics before diving into advanced topics like Reinforcement Learning from Human Feedback (RLHF). - The speaker recommends starting with building a simple model from scratch, like a small language model, to gain a concrete understanding of the underlying architecture and concepts. - The fast-paced nature of the AI industry, exemplified by the rapid evolution of reasoning models, makes it risky for beginners to only focus on the cutting edge without understanding fundamentals. - Academic research paths often involve deep theoretical work (like the concepts in the 'Bubbles and the End of Stagnation' book) while industry roles might focus more on practical application and engineering. - The speaker suggests that the community culture, especially in Silicon Valley, sometimes encourages an unsustainable '996' work ethic, leading to burnout, which should be avoided by maintaining work-life balance. - Understanding the mechanics of existing models and papers (like those listed in the table from the 'RLHF Book') is crucial for effective contribution and research. ![Screenshot at 03:03: The guest presents the structure of Sebastian Raschka's 'Build a Reasoning Model \(From Scratch\)' book, illustrating a four-stage process from LLM basics to advanced training techniques, emphasizing hands-on learning.](https://ss.rapidrecap.app/screens/dGqhTpsu_5Y/00-03-03.jpg) **Context:** This discussion features Lex Fridman interviewing an AI researcher (likely Sebastian Raschka, based on the visual slides) about the best path for beginners entering the field of AI and large language model development. The conversation centers on whether newcomers should immediately focus on advanced techniques like RLHF or first master the foundational engineering and mathematical principles of transformers and LLMs, touching upon the intense work culture prevalent in some parts of the industry. ## Detailed Analysis The core advice for beginners in AI is to start with fundamental engineering and mathematical concepts before tackling complex post-training methods like RLHF. The speaker suggests that attempting to keep up with the rapid pace of AI development without a strong base leads to burnout, citing the high-risk, high-reward nature of working in frontier AI labs versus academia. The guest references his book, 'Build a Reasoning Model (From Scratch),' which outlines a path starting with LLM basics, moving to performance measurement, reasoning without training, and finally reasoning with training, emphasizing building from scratch to understand the internals. He points out that many popular papers, like those detailing RLHF, are complex and often assume a deep understanding of the underlying architecture. Furthermore, the discussion touches upon the potentially unhealthy work culture ('996' schedule) prevalent in some tech hubs, contrasting it with the academic pursuit of fundamental research, which he finds more rewarding. He notes that while companies like OpenAI are pushing the state-of-the-art, the underlying mathematical principles remain crucial for long-term impact. ### Advice for Beginners - Prioritize learning the fundamentals of LLM engineering and math over immediately diving into advanced topics like RLHF - Avoid burnout by focusing on foundational understanding rather than chasing every new paper or trend. ### Learning Strategy - Recommended to build a simple model from scratch (e.g., a small LLM) to understand the core mechanics before trying to implement complex techniques. ### Industry Culture Comparison - Notes the high-pressure, high-risk environment in some frontier AI labs (like Silicon Valley companies) compared to the slower, more methodical pace often found in academia. ### The Role of RLHF - While important, RLHF is an advanced technique that requires a solid grasp of the base model architecture first; the speaker suggests understanding the math behind these concepts first. ### Book Reference - Highlights Sebastian Raschka's 'Build a Reasoning Model (From Scratch)' as a good resource that structures learning across foundational LLM basics, evaluation, reasoning without training, and training methods. ![Screenshot at 00:03: The guest displays the covers of his books, 'Build a Large Language Model' and 'Build a Reasoning Model' \(both from scratch\), highlighting his focus on foundational AI education.](https://ss.rapidrecap.app/screens/dGqhTpsu_5Y/00-00-03.jpg) ![Screenshot at 00:06: A diagram illustrating the multi-stage process for building a reasoning LLM, covering LLM basics, measuring performance, reasoning without training, and training with reinforcement learning.](https://ss.rapidrecap.app/screens/dGqhTpsu_5Y/00-00-06.jpg) ![Screenshot at 00:19: The guest describes his recommendation to start by implementing a simple model from scratch on one's own computer.](https://ss.rapidrecap.app/screens/dGqhTpsu_5Y/00-00-19.jpg) ![Screenshot at 03:04: A slide summarizing the complexity of the 'Transformers' library, stating it maintains 1M Python LOC and over 400 models, emphasizing the scale of the codebase.](https://ss.rapidrecap.app/screens/dGqhTpsu_5Y/00-03-04.jpg) ![Screenshot at 07:50: A slide referencing 'The RLHF Book' by Nathan Lambert, detailing its structure across introductions, problem setup, optimization tools, and advanced topics.](https://ss.rapidrecap.app/screens/dGqhTpsu_5Y/00-07-50.jpg)