Do LLMs Feel? Teaching Emotion Recognition with Prompts, Retrieval, and Curriculum Learning

Quick Overview

The researchers developed the PRC EMO framework, which combines prompting, retrieval, and curriculum learning to teach Large Language Models (LLMs) to recognize and interpret human emotions, achieving superior performance over previous methods, especially in capturing subtle emotional nuances in dialogue.

Key Points: The PRC EMO framework successfully teaches LLMs to recognize human emotions, moving beyond simple word prediction to understand emotionally nuanced dialogue. The framework utilizes a three-pronged approach: Prompting, Retrieval, and Curriculum Learning. The curriculum learning strategy involved a systematic climb from simple emotional exchanges to complex ones, starting with easy buckets like explicit emotion recognition (ERC) and progressing to weighted emotional shifts (WES). PRC EMO significantly outperformed older methods like pure prompt-based learning on the Iemocap dataset, achieving 71.95% accuracy on ERC and 70.4% on Mel. The researchers found that explicit emotion recognition (ERC) was easier for models to learn than weighted emotional shifts (WES), which require deeper contextual understanding. The new method generated external knowledge, like a 'cheat sheet' of past experiences, which helped the model interpret the impact of subtle emotional shifts in conversations.

Context: This video discusses research aimed at improving Large Language Models' (LLMs) ability to understand human emotion beyond just recognizing explicitly stated feelings. The work, presented by researchers including Rosalind Picard, focuses on a new framework called PRC EMO (Prompting, Retrieval, Curriculum Learning for Emotion Recognition), designed to teach models to interpret subtle, implicit emotional cues in conversation, a known limitation of earlier LLMs that often struggled with context and nuance.

Detailed Analysis

The presentation details the PRC EMO framework developed to enhance LLMs' capacity for emotionally intelligent interaction, moving past simple next-word prediction to truly human-like emotional understanding. The core of the method involves three components: Prompting, Retrieval, and Curriculum Learning. The curriculum is structured systematically, starting with easier tasks like explicit emotion recognition (ERC) examples (e.g., identifying if someone says they are happy or sad) and progressing to more challenging tasks like recognizing weighted emotional shifts (WES) across consecutive turns in a dialogue. The researchers demonstrated that the PRC EMO model significantly outperformed previous methods, including those based on models like GPT-4o, on the Iemocap benchmark for both ERC (71.95% vs. 69.7%) and Mel (70.4% vs. 64.7%). The researchers noted that while older models often failed to capture subtle, implicit emotional intent, the retrieval mechanism in PRC EMO allows the model to reference a large, diverse library of past interactions (36,700 utterances) to better inform its nuanced interpretations. This approach proved particularly effective in scenarios where speakers mask their true feelings, forcing the AI to analyze the context and potential underlying emotional dynamics.