# A Guide to LLMs in Modeling and Simulation: From Core Techniques to Critical Challenges

Source: https://www.youtube.com/watch?v=VBxN8eFMH6A
Recap page: https://rapidrecap.app/video/VBxN8eFMH6A
Generated: 2026-02-08T13:03:47.805+00:00

---
## Quick Overview

The paper argues that the scientific community is stuck in a mindset where they try to make Large Language Models (LLMs) deterministic by adding more prompts, when the better approach is to treat them as inherently probabilistic systems and use techniques like design of experiments to probe their behavior and manage uncertainty, especially concerning subtle changes in output due to minor input variations.

**Key Points:**
- The core argument is that LLMs should be treated as probabilistic, not deterministic, despite community efforts to force deterministic outputs through extensive prompting.
- The paper references research by Philip J. Gabinelli from Old Dominion University, who demonstrated that even simple graph structures exhibit variation when modeled probabilistically versus deterministically.
- Gabinelli's work shows that using a format like P(A|B) + P(C) results in slightly different final sums when the order of operations changes, highlighting inherent non-determinism.
- A key concept discussed is the 'interface bleeding problem,' where changing the hardware stack or using different LLMs results in output variation, even when the logic remains the same.
- The authors suggest that instead of constantly adding more prompts (which they liken to 'burying' the issue), researchers should use design of experiments (DoE) to systematically probe the model's behavior.
- The paper contrasts the reliability of probabilistic analysis (like using Alloy when code is wrong) versus relying solely on the LLM's internal, non-guaranteed deterministic behavior.
- The ultimate goal is to move away from seeking absolute scientific reproducibility in LLM outputs and instead focus on managing the inherent stochasticity.

![Screenshot at 00:10: The hosts introduce the topic: looking at a paper discussing the capabilities and modeling/simulation of LLMs, setting the stage for the discussion on determinism versus probability.](https://ss.rapidrecap.app/screens/VBxN8eFMH6A/00-00-10.jpg)

**Context:** This discussion centers on a research paper by Philip J. Gabinelli from Old Dominion University, focusing on the proper methodology for modeling and simulation using Large Language Models (LLMs). The context is a critique of the current approach within the scientific community, which often attempts to enforce deterministic, reproducible results from LLMs by adding layers of prompts, rather than acknowledging and managing the inherent probabilistic nature of these tools.

## Detailed Analysis

The discussion critiques the scientific community's tendency to treat LLMs as deterministic entities, often attempting to force specific, repeatable outputs through excessive prompting and engineering. The paper advocates for treating LLMs as fundamentally probabilistic systems. The speaker cites Philip J. Gabinelli's work, which shows that even simple mathematical operations (like A + (B + C) vs (A + B) + C) yield slightly different results when run through an LLM, demonstrating non-determinism even in basic logic. This variation is compounded when considering hardware differences (e.g., H100 vs A100 GPUs) or using different LLMs, leading to the 'interface bleeding problem.' The paper suggests that instead of adding more prompts or data to mask these issues, researchers should adopt a Design of Experiments (DoE) approach. This involves systematically varying inputs and analyzing the resulting statistical patterns (like the distributions of answers) to understand the model's true behavior, rather than seeking absolute, singular factual answers. The core takeaway is that for complex tasks, the context (like the prompt itself) dictates the output's frame of reference, and managing this stochasticity is more crucial than forcing an illusory deterministic result.

### Paper Introduction

- Discussion focuses on a paper by Philip J. Gabinelli from Old Dominion University concerning LLMs in modeling/simulation
- The paper examines what LLMs can and cannot do without serious engineering.

### The Problem of Determinism

- Gabinelli's argument is that the scientific community is stuck treating LLMs like magic wands; The core issue is the inherent probabilistic nature that simple prompting ignores.

### Mathematical Example

- A simple (A+B)+C vs A+(B+C) calculation shows small differences in output due to the LLM's internal processing, even when the logic is the same
- This is likened to faulty programming or a student ignoring the textbook for past quiz answers.

### Hardware and Context Influence

- The model's output can change based on hardware (H100 vs A100) and the context window size, leading to non-reproducible science
- This is termed the 'interface bleeding problem,' where the physical environment affects the logical output.

### Proposed Solution

- The paper suggests using Design of Experiments (DoE) to probe the model's behavior, looking at statistical patterns (like the distribution of answers) rather than single outputs
- This is contrasted with simply adding more prompts, which is deemed inefficient and masking.

### Conclusion and Analogy

- The final stage moves from the 'magic wand phase' to the 'engineering phase' where the goal is to manage uncertainty and understand the probabilistic nature of the output, not force deterministic facts.

![Screenshot at 00:00: Introductory screen featuring two podcasters over an oscilloscope graphic and the text 'Become a member today!'](https://ss.rapidrecap.app/screens/VBxN8eFMH6A/00-00-00.jpg)
![Screenshot at 00:26: The speaker begins explaining the paper's focus, noting that the approach feels like trying to move the whole conversation from programming logic to reality checks.](https://ss.rapidrecap.app/screens/VBxN8eFMH6A/00-00-26.jpg)
![Screenshot at 01:00: The speaker highlights the contrast between the naive approach \(like asking for a recipe\) and the paper's focus on simulation and risk assessment for high-stakes environments.](https://ss.rapidrecap.app/screens/VBxN8eFMH6A/00-01-00.jpg)
![Screenshot at 02:04: The speaker mentions a study highlighting graph data that showed the model's output was heavily influenced by the simple structure of nodes and edges, which is mathematical, not social simulation.](https://ss.rapidrecap.app/screens/VBxN8eFMH6A/00-02-04.jpg)
![Screenshot at 03:09: The speaker emphasizes that the proposed method involves decomposition and characterization of relationships, rather than just adding more prompts.](https://ss.rapidrecap.app/screens/VBxN8eFMH6A/00-03-09.jpg)