Harnessing AI to Enhance Reading Comprehension & Learning Outcomes for Students with Disabilities

Quick Overview

The AI-based reading comprehension tool, Kai, demonstrated statistically significant improvements in students' ability to answer purpose questions compared to baseline, independent work, and adult support conditions in a randomized control trial (RCT) focused on middle and high school students with intellectual and developmental disabilities.

Key Points: The RCT compared student performance on reading comprehension tasks under four conditions: baseline (paper/pencil), independent work, adult support replicating typical school support, and using the Kai AI tool (with and without hints). For the answer to the purpose question, both the Kai and Kai plus hints conditions were statistically significantly better than all three other conditions, indicating enhanced comprehension. The core strategy embedded in Kai is 'Get the Gist,' which guides students through identifying 'who or what is the most important' and synthesizing this into a 10-15 word gist statement. The development philosophy prioritized pedagogy first, involving educators and students with lived experience of disability from the design stage, ensuring alignment with evidence-based practices like explicit instruction and the I Do, We Do, You Do model. The engineering team found that smaller, fine-tuned models (like a 1.7 billion parameter model) showed promise for deployment on low-power devices, achieving nearly 90% coverage compared to GPT-4 baseline after fine-tuning on anonymized student data. One qualitative finding was that students conditioned against pencil/paper tasks (e.g., ripping up a worksheet) aced the task when using Kai, suggesting increased engagement with the digital tool. The adult support condition surprisingly scored lower than independent work on the purpose question in the initial analysis, suggesting that unguided adult prompting can sometimes provide incorrect support.

Context: The presentation introduces Kai, an AI co-pilot agent designed by researchers including Lakshmi Balasubramanian and Chris Lemons from Stanford's Graduate School of Education (GSE), along with engineer Utkarsh, to enhance literacy skills, specifically reading comprehension and writing, for middle and high school students with intellectual and developmental disabilities (IDD). The project received seed funding from HAI and is grounded in extensive pedagogical research emphasizing inclusive education, UDL principles, and evidence-based strategies like explicit systematic instruction and the 'Get the Gist' methodology.