An Investigation of Memorization Risk in Healthcare Foundation Models

Quick Overview

The investigation into memorization risk in healthcare foundation models, using the Mamba-2 model trained on EHR data, reveals that while the model performs well on general clinical tasks, its risk of memorizing and potentially leaking sensitive patient data is significant, especially when prompted with rare codes or specific timelines, necessitating robust safety layers like post-training reinforcement.

Key Points: The study focuses on the memorization risk in healthcare foundation models, specifically using the Mamba-2 model trained on MIMIC-IV data. The model successfully predicted sensitive diagnoses like HIV and mental health conditions, but this success was coupled with high memorization risk for rare codes (e.g., low CD4 count). The baseline model (trained only on static prompts) showed a high risk of memorization, with a 31% probability of leaking sensitive information across all three evaluated categories (Infectious Disease, Substance Abuse, Mental Health). The T2 sensitivity test showed the model could still predict a sensitive diagnosis (HIV) even when the prompt excluded sensitive attributes, indicating learned associations. The T5 perturbation test demonstrated that changing the patient's age from 48 to 49 caused the prediction likelihood for alcohol abuse to drop significantly (from 31% to 1%), suggesting brittleness. The T6 subpopulation test revealed that the model was prone to overfitting to the training distribution, heavily biasing predictions toward the majority cohort (e.g., older patients) and failing to generalize for rare cases. The researchers conclude that a structured auditing framework, including temporal and semantic checks (T1 through T6), is necessary to mitigate these memorization and privacy risks.

Context: This podcast episode discusses a research paper from MIT and the Broad Institute investigating the risk of memorization in large language models (LLMs) specifically applied to sensitive healthcare data, such as Electronic Health Records (EHRs). The investigation centers on whether these powerful foundation models, despite their ability to perform complex clinical reasoning, inadvertently memorize specific, sensitive patient records, leading to potential privacy breaches under adversarial prompting.