# Agentic Refactoring: An Empirical Study of AI Coding Agents Source: https://www.youtube.com/watch?v=I_ggx5FRdl8 Recap page: https://rapidrecap.app/video/I_ggx5FRdl8 Generated: 2025-11-14T17:11:18.341+00:00 --- ## Quick Overview Agentic refactoring, when done by AI coding agents, significantly improves code quality metrics like complexity and maintainability compared to human developers, with agents achieving a 52.5% reduction in variable type complexity and a 43% reduction in overall complexity across refactoring instances, although they still struggle with deep architectural changes. **Key Points:** - Agentic refactoring, where AI agents fix code, results in a 52.5% reduction in variable type complexity and a 43% reduction in overall complexity. - The study analyzed over 15,000 individual refactoring instances across various open-source Java projects. - AI agents were significantly more focused on surface-level cleanup (like renaming variables at 11.9%) than deep structural changes (only 1.1% of activity). - Human developers, conversely, dedicate only 24.4% of their refactoring time to low-level changes, prioritizing structural improvements. - The primary motivation for agentic refactoring was internal code quality, accounting for 86.9% of agent activity, while human motivation was slightly more balanced. - The study used a classifier to determine if refactoring was for immediate maintainability or deep architectural health, finding agents heavily favored the former. ![Screenshot at 00:25: The comparison between agent activity \(53.9% low-level changes\) and human developer activity \(only 24.4% low-level changes\) highlights the agents' focus on immediate, surface-level maintenance over deep architectural refactoring.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-00-25.png) **Context:** This video discusses the findings of an empirical study titled "Agentic Refactoring: An Empirical Study of AI Coding Agents," which investigated how AI agents perform code refactoring tasks compared to human developers. The core concept revolves around whether AI agents can effectively clean up code structure, which is often tedious, or if they only focus on superficial fixes like renaming variables, using metrics like cyclomatic complexity and code smell detection to evaluate impact. ## Detailed Analysis The empirical study on agentic refactoring demonstrated that AI coding agents are highly effective at localized, low-level code cleanup, leading to measurable improvements in code metrics, though they fall short on deep architectural restructuring. The agents achieved a substantial 52.5% reduction in variable type complexity and a 43% overall complexity reduction across over 15,000 refactoring instances. Specifically, 86.9% of agent activity was dedicated to immediate code quality improvements, such as renaming variables (11.9%) and extracting helper methods, which resulted in cleaner code that was easier to read and maintain. In contrast, human developers prioritize structural improvements, dedicating only 24.4% of their time to low-level changes. The study found that when agents addressed deep architectural issues, the success rate was minimal, with the primary limitation being an 'architectural blindness' where they failed to recognize deep flaws, even when explicitly instructed. The data showed that agents were far more likely to accept a refactoring request if it was explicitly stated (87% acceptance) than if it was implicit, suggesting a need for clearer instructions for complex tasks. ### Refactoring Impact Analysis - Agentic refactoring achieved 52.5% reduction in variable type complexity - 43% reduction in overall complexity - 86.9% of agent activity focused on immediate quality improvements ### Human vs. Agent Priorities - Humans spent 75.6% of time on structural changes while agents spent only 13.1% - Agents focused heavily on surface-level cleanup like renaming variables (11.9%) ### Key Metrics Affected - Structural metrics like class signatures and API changes showed minimal improvement from agents - Readability and complexity saw significant gains ### Agent Limitations - Agents showed an 'architectural blindness' - Deep structural refactoring (like code simplification) was rarely prioritized or successfully executed ### Acceptance Rates - Agents accepted 87% of explicitly stated refactoring requests but failed to reduce deep structural complexity significantly ![Screenshot at 00:13: The introduction of the concept of AI agents acting as autonomous teammates for code refactoring.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-00-13.png) ![Screenshot at 00:43: Mention of the empirical study 'Agentic Refactoring' used as the source for the analysis.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-00-43.png) ![Screenshot at 01:48: A visual representation of the difference in focus: agents focusing on surface-level code changes.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-01-48.png) ![Screenshot at 03:34: The key distinction between 'Code Janitor vs. Software Architect' refactoring roles.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-03-34.png) ![Screenshot at 06:06: The contrast in activity: agents focusing on easy maintenance versus deeper structural work.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-06-06.png) ![Screenshot at 07:34: The two metrics measured in the study: structural metrics and code smells.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-07-34.png) ![Screenshot at 08:00: The finding that agents failed to consistently reduce the number of code smells, suggesting a limitation in deep problem-solving.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-08-00.png) ![Screenshot at 09:55: The study provided agents with specialized analysis tools like the 'Design' tool to detect flaws.](https://ss.rapidrecap.app/screens/I_ggx5FRdl8/00-09-55.png)