# Beyond RAG - A system that understands your documents Source: https://www.youtube.com/watch?v=rMADSuus6jg Recap page: https://rapidrecap.app/video/rMADSuus6jg Generated: 2026-01-11T15:01:35.883+00:00 --- ## Quick Overview Agentic File Search, an approach that employs a three-phase strategy (Parallel Scan, Deep Dive, Backtrack) and specialized tools, overcomes the limitations of traditional RAG by enabling dynamic exploration, understanding document structure, and following cross-references, leading to more accurate answers for complex, multi-document queries. **Key Points:** - The standard RAG pipeline fails when documents are structured and rely on cross-references, as chunking breaks contextual relationships and makes references invisible to vector embeddings (0:23, 0:46). - Agentic File Search uses a Three-Phase Strategy: Phase 1 (Parallel Scan using scan_folder()) provides a quick preview of all documents; Phase 2 (Deep Dive using parse_file()) extracts information from relevant documents; and Phase 3 (Backtrack using parse_file() again) follows cross-references to skipped documents (4:04, 5:05, 15:18). - The system uses six core tools (scan_folder, preview_file, parse_file, read, grep, glob) governed by an Agent decision loop that uses Gemini 3 Flash with structured JSON output to decide which tool to use next (10:59, 12:24). - The key difference is that RAG relies on fixed, pre-computed embeddings and retrieves the same chunks regardless of query nuance, while Agentic Search uses dynamic exploration that adapts to each query and follows logical paths (4:47). - Parallel scanning via scan_folder is crucial because it provides the agent full context in a single API call, making it 4x faster than sequential processing (13:56, 14:25). - Structured output using JSON schema prevents parsing failures and ensures predictable actions, eliminating an entire class of bugs common in unstructured text retrieval (18:17). - The system is extensible, allowing users to add new tools by defining a function and updating the agent's tool dictionary and system prompt, or swap LLM providers by updating imports in agent.py (17:34, 17:47). ![Screenshot at 1:25: The 'Chunking Problem' slide illustrates how splitting a document into small chunks \(Chunk 47 and 48\) breaks the logical relationship needed to answer a query about purchase price adjustment methodology, a failure RAG exhibits \(1:23\).](https://ss.rapidrecap.app/screens/rMADSuus6jg/00-01-25.jpg) **Context:** The video contrasts the limitations of traditional Retrieval-Augmented Generation (RAG) systems, particularly when dealing with complex, interconnected documents, against a proposed Agentic File Search implementation called 'fs-explorer'. The core problem identified with RAG is that document chunking destroys the inherent structure and cross-references present in legal, technical, or research documents, leading to incomplete answers. The agentic approach aims to overcome this by dynamically exploring the document set using a set of specialized tools. ## Detailed Analysis The video argues that traditional RAG is insufficient for documents with complex structure and cross-references because the initial step of chunking destroys context and makes references invisible to the vector database (0:23). The proposed solution is Agentic File Search, which uses a three-phase strategy to reason over documents like a human would. Phase 1 is a 'Parallel Scan' using the `scan_folder()` tool to quickly get previews of all documents, providing the agent full context upfront (13:56). Phase 2 is a 'Deep Dive' using `parse_file()` only on relevant documents identified in Phase 1 to extract full content (15:15). Phase 3 is 'Backtrack', which uses `parse_file()` again to follow cross-references to documents that were initially skipped (15:18). The system architecture centers on an Agent that communicates with a Workflow Engine and Google Gemini 3 Flash, using structured JSON output for reliable decision-making (11:11, 12:10). The agent dynamically selects from six available tools (`scan_folder`, `preview_file`, `parse_file`, `read`, `grep`, `glob`) based on context, avoiding the fixed pipeline of RAG (11:40, 12:44). Key insights include that parallel scanning is crucial for speed and context, structured JSON output prevents parsing errors, and backtracking ensures thoroughness by following cross-references RAG would miss (18:07). The system is designed to be extensible for adding new tools or swapping LLM providers with minimal code changes (17:34). ### The Promise of RAG - Documents are converted to embeddings stored in a Vector DB, which the LLM uses to answer questions (0:03) ### The Reality of RAG - Chunks lose context, cross-references are invisible to embeddings, similarity is not relevance, and there is no reasoning (0:23) ### The Chunking Problem - Documents are structured, but chunking destroys that structure, breaking relationships between sections (1:39) ### Cross-References Are Everywhere - RAG retrieves one chunk, misses the chain of references necessary for complex queries (1:50, 2:07) ### Agentic File Search - Replaces fixed retrieval with dynamic exploration that scans all documents in parallel (3:34, 3:46) ### Three-Phase Strategy - 1. Parallel Scan (scan_folder) for previews; 2. Deep Dive (parse_file) on relevant docs; 3. Backtrack (parse_file again) for skipped references (4:04, 15:18) ### Key Differences (RAG vs. Agentic Search) - RAG uses fixed embeddings and same chunks retrieved regardless of query nuance; Agentic Search uses dynamic exploration, adapts to each query, and follows logical paths (4:37) ### Core Components - Workflow Engine (LlamaIndex Workflows, async execution), Agent (Gemini 3 Flash, structured JSON output), and Document Parser (Docling library for PDFs, DOCX, etc.) (11:55, 12:22) ![Screenshot at 0:03: The initial RAG pipeline diagram shows Documents -\> Embeddings -\> Vector DB -\> LLM \(0:03\)](https://ss.rapidrecap.app/screens/rMADSuus6jg/00-00-03.jpg) ![Screenshot at 0:26: The 'Reality' slide highlights RAG limitations: chunks lose context, cross-references are invisible, and similarity misses logical connections \(0:29\)](https://ss.rapidrecap.app/screens/rMADSuus6jg/00-00-26.jpg) ![Screenshot at 1:19: The 'Chunking Problem' demonstrates how query context is lost when documents are split into chunks \(e.g., Chunk 47 and 48\) \(1:20\)](https://ss.rapidrecap.app/screens/rMADSuus6jg/00-01-19.jpg) ![Screenshot at 3:35: The Agentic File Search flow starts with User Query, scans all docs in parallel, deep dives into relevant docs, and follows references \(3:35\)](https://ss.rapidrecap.app/screens/rMADSuus6jg/00-03-35.jpg) ![Screenshot at 17:06: The Cross-Reference Detection agent behavior explicitly shows detecting references, checking if skipped, and backtracking to parse the skipped file \(17:07\)](https://ss.rapidrecap.app/screens/rMADSuus6jg/00-17-06.jpg)