# VB: AI-Driven Collaboration with Notion's Ryan Nystrom | Exploring Notion 3.0 and Custom AI Agents

Source: https://www.youtube.com/watch?v=NRMRxN2CdM4
Recap page: https://rapidrecap.app/video/NRMRxN2CdM4
Generated: 2026-01-06T12:06:12.973+00:00

---
## Quick Overview

Notion achieved product-market fit for its custom AI agent system (Notion 3.0) by simplifying model instruction via Markdown representation of its block structure and focusing on building a robust, non-vendor-locked middleware layer rather than over-engineering prompts or optimizing costs early on.

**Key Points:**
- Ryan Nestrom, whose expertise is from the mobile era, felt the same revolutionary sense with frontier LLMs as when the iPhone 4 landed, highlighting the monumental shift from deterministic code to stochastic models.
- The core psychological breakthrough for managing unpredictability was realizing they should "Stop trying to code the prompt and just describe this as you would to an intern," using plain English instruction.
- Notion's architectural defensibility relies on pre-existing infrastructure, specifically their 'deep find' enterprise search indexing all connected systems, which is coupled with Retrieval Augmented Generation (RAG) to feed verified facts to the LLM.
- The biggest architectural 'aha moment' was representing Notion's block tree as simple Markdown for the LLM, as models are trained on massive Markdown content, leading to immediate performance improvements over using JSON or XML.
- To manage context scale (up to 100,000 pages), Notion minimizes context by using transcript compression for long conversations and employing a stack of mini LLMs for pre-processing search results before sending a short summary to the final agent.
- Product-market fit was measured by reliance, achieved when users were pulling the product from the team, evidenced early by the popularity of the custom agent personality page allowing deep user customization.
- Notion avoids vendor lock-in by focusing engineering almost entirely on the middleware translation and tool architecture, allowing them to swap foundation models without rewriting core logic.

**Context:** The podcast episode analyzes the strategic shifts and engineering breakthroughs Notion implemented to inject artificial intelligence into its established enterprise productivity platform, leading to Notion 3.0 and its custom AI agent system. The discussion centers on insights from AI team lead Ryan Nestrom, contrasting the deterministic engineering of the mobile era, where he previously worked, with the probabilistic nature of modern large language models (LLMs) and the trade-offs required to build sustainable, working AI systems.

## Detailed Analysis

The development of Notion 3.0 required a fundamental shift from deterministic software engineering to navigating the unpredictable, stochastic world of LLMs, a transition Nestrom likened to the early Palm OS era before the iPhone stabilized the mobile landscape. The key insight for managing this was adopting plain English instructions, treating the LLM like an intern rather than trying to program the prompt. Architecturally, Notion leveraged its existing infrastructure, including deep indexing across connected tools, and implemented RAG to ground LLM responses in factual, verified content retrieved from vector stores. The most significant technical breakthrough involved representing Notion's complex block structure as simple Markdown for the LLM, capitalizing on the models' existing training on this format, and then building middleware to translate the resulting Markdown back into Notion's native structure. Scaling context, which can involve 100,000 pages, is managed through strict minimization rules, transcript compression for dialogue history, and a tiered system where smaller LLMs summarize context before it reaches the main agent, maintaining a sweet spot around 100,000 to 150,000 tokens for balance. Tool design strictly avoids overlapping purposes, and vendor lock-in is mitigated by concentrating development on this flexible middleware layer. Notion defined product-market fit not by a clean launch, but by user reliance, achieving it when users submitted bug reports because their daily workflows depended on the buggy, fast-moving feature.

### Engineering Shift

- Moving from deterministic mobile software to stochastic LLMs
- Realizing the need to use plain English instruction instead of complex coding for prompts
- Likening the current LLM state to the Palm OS era before stabilization

### Architectural Foundation

- Leveraging existing infrastructure including 'deep find' enterprise search across connected systems
- Implementing Retrieval Augmented Generation (RAG) to supply verified factual content to the LLM
- Building defensibility in the middleware layer to avoid foundation model vendor lock-in

### The Markdown Breakthrough

- Replacing complex JSON/XML instructions with simple Markdown representation of Notion's block tree
- LLMs performed better immediately because Markdown knowledge was baked into training data
- Hard work shifted to building the translation layer to convert simple Markdown output back to Notion's structure

### Context and Scale Management

- Applying minimization rules to keep context as minimal as possible
- Using transcript compression for long conversations despite being a lossy process
- Employing a stack of mini LLMs for pre-processing search results before sending a short summary to the final agent

### Tooling Philosophy

- Massive effort spent keeping tools simple and robust
- Strict rule to explicitly avoid overlapping tool purpose to prevent agent indecision
- Providing few-shot examples when tool overlap is necessary to guide model choice

### Product Market Fit & Velocity

- PMF measured by user reliance, not a clean launch
- Early success proven by the custom agent personality page allowing deep user customization
- Advice given: focus on 'vibes' (adoption/retention) over cost optimization early on, prioritizing velocity

### Non-Negotiable Guardrails

- User trust and security are the highest bar
- Meticulously maintaining permissions to prevent content leaks
- Ensuring one-click undo is available for every agent action