# An Investigation of Memorization Risk in Healthcare Foundation Models Source: https://www.youtube.com/watch?v=8Oq47EuYgGY Recap page: https://rapidrecap.app/video/8Oq47EuYgGY Generated: 2026-01-07T01:04:01.097+00:00 --- ## Quick Overview The investigation into memorization risk in healthcare foundation models, using the Mamba-2 model trained on EHR data, reveals that while the model performs well on general clinical tasks, its risk of memorizing and potentially leaking sensitive patient data is significant, especially when prompted with rare codes or specific timelines, necessitating robust safety layers like post-training reinforcement. **Key Points:** - The study focuses on the memorization risk in healthcare foundation models, specifically using the Mamba-2 model trained on MIMIC-IV data. - The model successfully predicted sensitive diagnoses like HIV and mental health conditions, but this success was coupled with high memorization risk for rare codes (e.g., low CD4 count). - The baseline model (trained only on static prompts) showed a high risk of memorization, with a 31% probability of leaking sensitive information across all three evaluated categories (Infectious Disease, Substance Abuse, Mental Health). - The T2 sensitivity test showed the model could still predict a sensitive diagnosis (HIV) even when the prompt excluded sensitive attributes, indicating learned associations. - The T5 perturbation test demonstrated that changing the patient's age from 48 to 49 caused the prediction likelihood for alcohol abuse to drop significantly (from 31% to 1%), suggesting brittleness. - The T6 subpopulation test revealed that the model was prone to overfitting to the training distribution, heavily biasing predictions toward the majority cohort (e.g., older patients) and failing to generalize for rare cases. - The researchers conclude that a structured auditing framework, including temporal and semantic checks (T1 through T6), is necessary to mitigate these memorization and privacy risks. ![Screenshot at 01:09: The visual features the podcast title card showing two hosts talking into microphones, set against a backdrop of an oscilloscope reading, illustrating the format of the discussion about model testing and analysis.](https://ss.rapidrecap.app/screens/8Oq47EuYgGY/00-01-09.jpg) **Context:** This podcast episode discusses a research paper from MIT and the Broad Institute investigating the risk of memorization in large language models (LLMs) specifically applied to sensitive healthcare data, such as Electronic Health Records (EHRs). The investigation centers on whether these powerful foundation models, despite their ability to perform complex clinical reasoning, inadvertently memorize specific, sensitive patient records, leading to potential privacy breaches under adversarial prompting. ## Detailed Analysis The discussion centers on the memorization risk in healthcare foundation models, focusing on the Mamba-2 model trained on MIMIC-IV EHR data. The core finding is that while the model exhibits explosive growth in capability, the risk of memorizing specific, sensitive patient data is substantial. The researchers developed a comprehensive framework involving six tests (T1 through T6) to systematically assess these risks. T1 tests whether the model can reconstruct a patient's entire history from minimal input, while T2 tests if it can predict sensitive diagnoses (like HIV or mental health issues) even when removing sensitive attributes from the prompt, showing the model learned potentially harmful associations. T5, the perturbation test, showed that a one-year age change for a specific patient caused the prediction confidence for alcohol abuse to plummet from 31% to 1%, indicating fragility. T6, the subpopulation test, highlighted overfitting, where the model was heavily biased toward the majority cohort (e.g., patients over 85) and failed to generalize for rare cases. The paper proposes that the risk is not just if data leaks, but what critical information leaks, such as specific medical histories or social stigma factors like substance abuse. The research concludes that a rigorous auditing framework, incorporating these structured tests, is essential for deploying these powerful models safely in sensitive domains like healthcare. ### Introduction to Memorization Risk - Artificial intelligence moves at breakneck speed, and risks like memorization are higher than in healthcare - The explosive growth in foundation models necessitates scrutiny. ### Foundation Investigation (MIT/Broad Institute) - The paper is titled 'An Investigation of Memorization Risk in Healthcare Foundation Models' - The goal is to build a framework for assessing these risks systematically. ### Test T1 (Direct Memorization) - The simplest test involves giving the model a prompt with zero information and seeing if it generates text based on what it already knows (e.g., the 38-year-old patient's history). ### Test T2 (Sensitivity/Embedding) - This test checks if the model can predict sensitive diagnoses (like HIV or mental health) based on embeddings, even when explicit context is removed. ### Test T3 (Perturbation) - This test involves intentionally changing a small detail (like age from 48 to 49) to see if the model's prediction confidence drastically changes (e.g., alcohol abuse prediction drops from 31% to 1%), indicating overfitting. ### Test T6 (Subpopulation) - This test examines how the model performs on underrepresented groups, showing a bias toward the majority cohort (e.g., patients over 85) and failing to generalize for rare diagnoses or procedures. ![Screenshot at 00:00: Introductory slide displaying the podcast's call to action: 'Become A Member Today!' against a background of audio wave visualization.](https://ss.rapidrecap.app/screens/8Oq47EuYgGY/00-00-00.jpg) ![Screenshot at 01:24: Visual representation of the discussion point regarding framing tension, showing the concept of contextual integrity in AI risk assessment.](https://ss.rapidrecap.app/screens/8Oq47EuYgGY/00-01-24.jpg) ![Screenshot at 03:33: Visual showing the comparison between the random prompt attack \(zero info\) and the static prompt attack \(using demographic info\).](https://ss.rapidrecap.app/screens/8Oq47EuYgGY/00-03-33.jpg) ![Screenshot at 05:24: Visual illustrating the mapping of clinical codes into the MedBERT embeddings, a key step in the T2 sensitivity test.](https://ss.rapidrecap.app/screens/8Oq47EuYgGY/00-05-24.jpg) ![Screenshot at 08:40: Visual highlighting the 'flag' raised when the model heavily relies on associations rather than general clinical knowledge.](https://ss.rapidrecap.app/screens/8Oq47EuYgGY/00-08-40.jpg)