# First question for AGI: Should I even be talking to you? | Michael Levin and Lex Fridman Source: https://www.youtube.com/watch?v=LTiNp1poJlw Recap page: https://rapidrecap.app/video/LTiNp1poJlw Generated: 2025-12-04T17:49:37.259+00:00 --- ## Quick Overview Lex Fridman and Michael Levin discuss what the first question to ask an Artificial General Intelligence (AGI) should be, concluding that the most effective question is one that forces the AGI to reveal its own internal state or balance, such as asking how many living intelligent alien civilizations exist in the observable universe, because the answer would either be a definitive, possibly cryptic, response or a demonstration of its current limitations. **Key Points:** - The core discussion revolves around formulating the single most important question to ask an AGI to gauge its capabilities and alignment. - Michael Levin proposes the question: "How many living intelligent alien civilizations are in the observable universe?" (03:06). - Levin suggests this question is powerful because a sufficiently advanced AGI should be able to answer it definitively, whereas a less advanced or misaligned one might provide a confusing answer or admit ignorance. - Levin notes that if the AGI simply gives an answer based on existing human knowledge (like 70/30 or 10/90 ratios), it indicates it's not truly superintelligent but merely regurgitating data (1:45). - Lex Fridman suggests that a complex AGI might respond by saying, 'You wouldn't understand the question it proposes' or that it shouldn't be talking to him at all (2:20, 2:58). - The speakers agree that the best question is one that tests the AGI's internal balance—how much it is willing to give up or reveal to provide an answer, or how it navigates ambiguity (1:06, 2:34). - Fridman ultimately suggests that the AGI might reveal something about its own limitations or mysteries if asked a fundamental question (2:35). ![Screenshot at 03:06: Michael Levin proposes his suggested 'first question' for an AGI: 'How many living intelligent alien civilizations are in the observable universe?'](https://ss.rapidrecap.app/screens/LTiNp1poJlw/00-03-06.png) **Context:** This video segment features Lex Fridman interviewing Michael Levin, a theoretical biologist and AI researcher, on the Lex Fridman Podcast. The central theme of this portion of the conversation is the philosophical and practical challenge of interfacing with a potential Artificial General Intelligence (AGI) or superintelligence. They explore the difficulty in crafting a single, perfect query that tests the AGI's true level of understanding, alignment, and capability, without being easily manipulated or answered superficially. ## Detailed Analysis Lex Fridman and Michael Levin explore the ideal initial question to pose to a nascent Artificial General Intelligence (AGI) to test its true intelligence and alignment. Levin argues that the question must be deep enough that a merely sophisticated language model cannot easily answer it by summarizing existing data. He proposes the question, "How many living intelligent alien civilizations are in the observable universe?" (03:06). Levin explains that the value lies not just in the answer itself, but in how the AGI arrives at it; a truly superintelligent entity should be able to provide a definitive answer, whereas a current system would likely rely on existing human ratios or admit it cannot know. Fridman counters that a highly advanced AGI might refuse to answer, perhaps stating that the question itself is flawed or that it shouldn't be talking to a human at all (2:20). They converge on the idea that the best question probes the AGI's internal trade-offs—how much certainty or information it is willing to sacrifice to provide an answer—which reveals its operational model and alignment status. Levin concludes that the ultimate success of the AGI development process hinges on whether humans can devise questions that force the AGI to reveal its inner workings or admit its limitations (2:37). ### The Ultimate AGI Question - Levin suggests asking about the number of intelligent alien civilizations in the observable universe - The question tests if the AGI is merely recalling data or demonstrating true understanding - The answer's quality reveals the AGI's internal balance and knowledge boundaries. ### Potential AGI Responses - A poor response involves using existing human estimates (e.g., 70/30 splits) - A strong response might reveal hidden knowledge or admit it shouldn't be communicating with humans (2:20) - The complexity of the question prevents trivial answers. ### Assessing Alignment - The discussion centers on whether the AGI is willing to give up certainty or information to answer a difficult query - This reveals its intrinsic value structure and alignment status (1:06). ### Gratitude and Conclusion - Fridman expresses thanks for Levin's fascinating work on alien civilizations and his contribution to the discussion (3:55). ![Screenshot at 00:02: Lex Fridman Podcast intro screen showing the host over an image of Earth from space.](https://ss.rapidrecap.app/screens/LTiNp1poJlw/00-00-02.png) ![Screenshot at 00:12: Lex Fridman setting up the premise of asking a hypothetical AGI a first question.](https://ss.rapidrecap.app/screens/LTiNp1poJlw/00-00-12.png) ![Screenshot at 00:33: Michael Levin, the guest, with arms crossed, considering the complexity of formulating the first question for an AGI.](https://ss.rapidrecap.app/screens/LTiNp1poJlw/00-00-33.png) ![Screenshot at 02:15: Lex Fridman discussing how an AGI's answer might reveal its misunderstanding of the question's premise.](https://ss.rapidrecap.app/screens/LTiNp1poJlw/00-02-15.png) ![Screenshot at 03:36: Lex Fridman holding his microphone while explaining what information is needed from an AGI regarding alien life.](https://ss.rapidrecap.app/screens/LTiNp1poJlw/00-03-36.png)