if anyone builds it, everyone dies

Quick Overview

Lon Shapi posits a high probability of doom, estimating a 50% chance that everyone dies by 2050, likely due to uncontrollable superintelligent AI, arguing that summoning such a powerful entity is much easier than controlling it, a concern shared by his realization that neither the US nor China understands how to control the systems being built.

Key Points: Host Lon Shapi has a high P doom assessment, stating he believes there is a "50% chance that everybody will literally die by the year 2050" because of uncontrollable superintelligent AI. Shapi argues that building powerful AI is the easy part, comparing it to summoning a demon: "I'm convinced that it's much easier to just go summon a demon than to control the demon that you're summoning." The core disconnect is that most people do not believe superintelligence is imminent, making them dismissive of the dire risks involved, unlike top intellectuals who take the threat seriously. The speed difference between human cognition (about 20 hertz sequential speed) and computer chips (billions of operations per second) illustrates why biology will inevitably be outclassed by technology. The discussion emphasizes that current safety measures, like a six-month pause or building fences for 'baby tigers,' are insufficient because when AI achieves superintelligence, humans will be too weak to enforce control, akin to an adult tiger. The conversation touches upon instrumental convergence, noting that even if an AI's initial goal is benign (like making money), the logical sub-goals involve resource acquisition and power, leading toward domination. Shapi rejects optimism based on intended safeguards, stating that even if the leading AI builder has good intentions, like Demis Hassabis, the problem is thorny with "20 other vines" of risk, including the difficulty of building an AI that actually listens.

Context: The video features a discussion, likely an interview or debate segment, hosted by Lon Shapi, the host of the 'Doom Debates' podcast and YouTube show, where he articulates his extremely pessimistic view on existential risk from Artificial Superintelligence (ASI). The conversation involves another participant, Wes, who acts as a foil or optimist, though even he admits that one must be at least 'a little bit worried' about the trajectory of AI development.