The Department of War is making a huge mistake.

Quick Overview

The Department of War's action against Anthropic, threatening to destroy it as a business for refusing to remove red lines against mass surveillance and autonomous weapons, highlights a dangerous precedent where the government seeks to coerce private companies into facilitating morally objectionable actions, mirroring authoritarian control systems despite the race with China.

Key Points: The Department of War designated Anthropic a supply chain risk because the company refused to remove red lines against using its models for mass surveillance and autonomous weapons. The speaker argues that while the government has the right to refuse Anthropic's models, threatening to destroy Anthropic as a private business for refusing to sell on the government's commanded terms sets a dangerous precedent, similar to a private contractor holding a "kill switch" on critical technology. The ability to enforce mass surveillance is rapidly approaching, as processing every camera feed in America could cost only $30 billion this year, dropping to $300 million by 2029 due to AI cost reductions. The speaker questions if the US is racing China only to adopt the "most ghoulish parts" of authoritarian systems by coercing companies into morally objectionable services. The core issue is determining to whom powerful AI systems should be aligned, contrasting the military's demand for obedience (which resembles successful technical alignment) against historical examples where refusing immoral orders saved lives, such as Stanislav Petrov's decision during the 1983 Soviet nuclear false alarm incident. Anthropic's advocacy for extensive regulatory apparatus, akin to nuclear or financial regulation, is deemed naive because vague terms like "catastrophic risk" or "threats to national security" are easily abused by future power-hungry leaders to enforce compliance. The analogy of AI to nuclear weapons breaks down because AI is a general-purpose transformation like industrialization, which societies regulated through banning specific destructive end uses, not by giving the government absolute control over the underlying technology.