this experiment could END the AI hype

Quick Overview

The ProFIT framework, which uses LLMs within an evolutionary feedback loop to generate and test trading strategies against historical data, significantly outperformed human-written strategies and random baselines in the Alpha Arena Season 1.5 competition, securing the top spot with a 12.11% aggregate return over two weeks, while most other models lost money.

Key Points: The Mystery Model (ProFIT framework) won Alpha Arena Season 1.5 with a 12.11% aggregate return over two weeks, while most other LLMs lost money. ProFIT is a large-language-model-driven evolutionary search framework that embeds LLMs within an evolutionary feedback loop, allowing them to iteratively improve trading strategies. The framework tests generated code modifications against historical data (backtesting) and uses a Minimum Acceptance Score (MAS) to filter viable candidates, avoiding local optima found in previous self-improving systems. In the competition, 32 instances of LLMs managed $320,000 in real capital, ingesting news, indices, and microstructure data every 6 minutes. The paper highlights that while LLMs excel at memorization tasks (like academic benchmarks), they struggle in complex, non-stationary environments like financial markets, which Alpha Arena is designed to simulate. The ProFIT framework consistently outperformed random and Buy-and-Hold strategies across seven liquid futures assets, achieving a mean annualized return increase of +44.21% over the seed strategy. The video concludes by posing the question of whether this approach will continue to scale or if the market conditions will change, making the historical data-based optimization obsolete.

Context: The video discusses the results of Alpha Arena Season 1.5, a competition where large language models (LLMs) were tasked with developing profitable trading strategies using real capital. The speaker analyzes why the winning model, named 'Mystery Model' (powered by the ProFIT framework), succeeded where others failed, particularly highlighting the limitations of standard LLM benchmarks and the novelty of ProFIT's evolutionary, self-improving code generation approach.