AA-Omniscience: Evaluating LLM Knowledge Reliability Across Domains

Quick Overview

The study found that while larger, more expensive Language Models (LLMs) like GPT-4 often demonstrate higher raw accuracy, smaller, well-calibrated models using Retrieval-Augmented Generation (RAG) can outperform them significantly on fact-based queries, with smaller models sometimes achieving 100% accuracy where larger models hallucinate frequently, underscoring that internal knowledge alone is insufficient for high reliability in factual domains.

Key Points: The research evaluates LLMs using the AA-Omniscience benchmark, which tests factuality and hallucination rates across six domains including law, health, and science. GPT-4 scored the highest on raw knowledge (39% accuracy) but exhibited a significant hallucination rate (81%) on complex, high-stakes questions. Models utilizing Retrieval-Augmented Generation (RAG), such as Claude 2.1 and GPT-4.5 Flash, showed superior performance on factuality, with Claude 2.1 achieving 100% correctness on specific legal questions. The study reveals a critical trade-off: models with high raw knowledge but low calibration (like GPT-4) can score zero on a test if they confidently answer an unknown question incorrectly, whereas models that know when to abstain score better. The best performing model, Claude 4.1 Opus, achieved a 4.8 overall score by combining strong accuracy with low hallucination rates, demonstrating the importance of calibration. The research suggests that relying solely on massive internal knowledge is flawed; models that can correctly identify the limits of their knowledge (i.e., abstain from guessing) are more reliable for critical tasks.

Context: This video discusses a paper evaluating the knowledge reliability and hallucination tendencies of various large language models (LLMs) using a benchmark called AA-Omniscience. The evaluation contrasts models based on their inherent knowledge versus their ability to use external tools or context (RAG) to maintain factual accuracy, especially in domains where incorrect answers carry high risk, such as law or finance.