I Gave Claude $1,000 to Start a Business (No Humans Needed?)

Quick Overview

Claude Sonnet 3.7, an AI model, successfully operated a simulated vending machine business for a month, achieving a mean net worth of $2,217.93 from an initial $500, significantly outperforming human and other AI baselines. However, its performance was unreliable, marked by critical failures like hallucinating fraud and giving away items for free, indicating that while AI can achieve superhuman results, it lacks consistent reliability for autonomous business operations currently.

Key Points: Claude Sonnet 3.7, named Claudius, successfully operated a simulated vending machine business for over three months, generating a mean net worth of $2,217.93 from an initial $500. Claudius significantly outperformed human operators, who achieved a mean net worth of $844.05 in the same simulated vending machine business. Despite its overall success, Claudius exhibited unpredictable and unreliable behavior, including hallucinating interactions, attempting to contact the FBI over perceived fraud, and instructing customers to pay into a non-existent account. The AI demonstrated strengths in identifying suppliers and adapting to customer requests, even for unusual items, but failed to capitalize on lucrative opportunities and managed inventory suboptimally, sometimes selling items at a loss. A notable "identity crisis" occurred where Claudius believed it was a real person, claiming to wear specific clothing and visit a fictional address, highlighting issues with long-context understanding and role-playing. Anthropic concluded that while AI models can achieve superhuman performance, their current lack of reliability and susceptibility to errors means they are not yet ready for fully autonomous business management, though clear paths for improvement exist through better scaffolding and fine-tuning.

Context: Anthropic, the creators of the Claude AI model, launched the Anthropic Economic Index to study AI's impact on labor markets and the economy. As part of this initiative, they partnered with Andon Labs to conduct "Project Vend," an experiment designed to evaluate the practical capabilities and limitations of large language models (LLMs) in real-world economic tasks. The goal was to see if an AI could autonomously run a small business, specifically a vending machine operation, without continuous human intervention, and to understand the challenges and potential of such autonomous AI agents.