Language Models Exhibit Inconsistent Biases Towards Algorithmic Agents and Human Experts

Quick Overview

Research reveals that Large Language Models (LLMs) exhibit significant, inconsistent biases favoring synthetic data or human experts depending on the prompt framing, with models trained on massive human-generated text showing a strong preference for human judgments (90% accuracy vs. 50% for the AI in one test), highlighting a fundamental challenge in aligning AI outputs with objective reality when human biases are embedded in the training data.

Key Points: LLMs exhibit inconsistent biases, favoring synthetic data or human experts based on prompt framing. In one test, the AI's stated preference for human experts (90% accuracy) was the inverse of its performance (50% accuracy) when forced to choose between them. The study compared models trained on massive human corpuses against those trained on synthetic data, specifically testing on high-stakes domains like air traffic control and medical diagnosis. The newer GPT-4 and Llama 3 70B models showed a massive algorithmic aversion bias, favoring synthetic data (70% accuracy) when the prompt was framed as an 'AI agent' task. When the prompt was switched to 'human expert' vs. 'algorithm' for the same task, the bias flipped, with models favoring the human expert's judgment (90% accuracy vs. 50% for the AI). The research suggests that the reliance on human-generated text in training leads to an inherent bias where models value human judgment over empirical data when the prompt suggests it.

Context: The discussion centers on a recent research paper analyzing the inherent biases exhibited by Large Language Models (LLMs) when asked to evaluate competing sources of information: synthetic data generated by other AIs versus judgments provided by human experts. The core concept being explored is how the foundational training data—billions of tokens of human-generated text—influences the model's preference, especially when dealing with high-stakes, objective tasks like medical diagnosis or tactical decisions like air traffic control.