A Proof-of-Concept for Explainable Disease Diagnosis Using LLMs and Answer Set Programming

Quick Overview

McCoy's hybrid architecture, combining Large Language Models (LLMs) with Answer Set Programming (ASP) via a component called a Grounder, successfully creates a robust and explainable system for medical diagnosis by ensuring the LLM's reasoning path is strictly constrained by verifiable logical rules, thus avoiding hallucinations and maintaining clinical rigor.

Key Points: McCoy proposes a hybrid architecture using LLMs and Answer Set Programming (ASP) to overcome the single largest barrier to AI use in high-stakes fields like medicine: trust. The system utilizes a 'Grounder' component to transform unstructured medical literature and patient data (symptoms, test results, history) into a formal, structured, and verifiable set of logical rules for the ASP solver. The LLM's core role is translation—converting natural language inputs into the logic engine's required format—rather than direct diagnosis. The ASP solver, constrained by these explicit rules, generates a diagnosis that is guaranteed to be consistent with the provided facts and logic, ensuring auditability and minimizing hallucination risk. Preliminary testing on a small set of diseases (like chickenpox) showed the system could correctly derive diagnoses based on symptoms, validating the approach against a known knowledge base. The key advantage over pure LLMs is that this framework forces the system to adhere to explicit logic, preventing it from generating plausible but potentially harmful incorrect diagnoses based on insufficient data.

Context: The video discusses a novel approach to creating trustworthy Artificial Intelligence systems for critical applications, specifically medical diagnosis, where the opacity and tendency of Large Language Models (LLMs) to 'hallucinate' create a massive barrier to adoption. The speaker introduces a proof-of-concept developed by someone named McCoy that integrates LLMs with symbolic AI, specifically Answer Set Programming (ASP), to enforce logical rigor and explainability on the AI's output.