Grounded by Experience: Gen Healthcare Prediction Augmented with Hierarchical Agentic Retrieval

Quick Overview

The Hierarchical Agentic Retrieval (HAR) system outperforms traditional Large Language Models (LLMs) in medical reasoning tasks, such as predicting hospital readmission, by employing two specialized agents—Agent Top (the primary physician) and Agent Low (the consultant)—that work collaboratively to ensure answers are grounded in evidence and contextually relevant, thereby avoiding hallucinations and improving accuracy across rare disease predictions.

Key Points: The research tackles the challenge of making LLMs trustworthy in high-stakes areas like healthcare prediction. The proposed solution is the Hierarchical Agentic Retrieval (HAR) system, which uses two agents: Agent Top (primary decision-maker) and Agent Low (consultant/retriever). HAR solves the problem of LLMs hallucinating by forcing the agents to collaborate and ground responses in evidence, unlike models that rely solely on internal parameters. In tests predicting hospital readmission (a 2-day prediction), HAR scored 9/10 correct, significantly outperforming baseline models that scored 3/10. Agent Top is rewarded for accuracy, while Agent Low is rewarded for quality and efficiency (fewer steps/less latency) in retrieving relevant sub-graphs of knowledge. This coordinated approach, leveraging both reasoning and retrieval, ensures the final answer is both accurate and ethically sound, avoiding bias toward common diseases. The system's success is demonstrated by its ability to handle rare conditions where large datasets often fail older, non-hierarchical models.

Context: The video discusses a research paper presenting the Hierarchical Agentic Retrieval (HAR) system, designed to improve the trustworthiness and accuracy of Large Language Models (LLMs) when applied to critical tasks like healthcare predictions. The core issue addressed is the tendency of standard LLMs to 'hallucinate' or produce plausible but incorrect answers, especially when dealing with rare data points, which is unacceptable when patient lives are at stake.