When LLM-Generated Refactorings Ripple: An Empirical Study of Repository Adoption Readiness
To address this gap, we conduct 1,500 generation attempts across 150 production files from 15 open-source Java repositories, five LLM configurations, and two dependency-aware generation conditions: single-prompt refactoring and LangGraph-based multi-agent refactoring.
ProofPaper ↗
Key points
- Large Language Models (LLMs) are increasingly used to refactor source files, but file-local evaluations do not reveal changes to contracts, dependencies, or caller assumptions affecting repository adoption.
- We evaluate these candidates using local changes, repository-facing structural indicators, detected caller impact, the behavioural-equivalence indicator, and available test evidence, and categorize them as Low-risk, Review-needed, High Contract/Dependency Risk, or Inconclusive.
- The results show that 1,365 candidates (93.8%) are classified as High Contract/Dependency Risk, whereas 106 (7.3%) have at least one detected broken downstream caller and 290 (19.9%) have a false behavioural-equivalence indicator; only eight satisfy the Low-risk criteria.
- These findings suggest that practitioners should treat LLM-generated refactorings as candidates requiring staged repository-level verification, while future research should validate screening and extraction precision and strengthen compilation, behavioural, and coordinated multi-file verification.
Sources (1)
- [1]When LLM-Generated Refactorings Ripple: An Empirical Study of Repository Adoption ReadinessarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 05:29 PM
To address this gap, we conduct 1,500 generation attempts across 150 production files from 15 open-source Java repositories, five LLM configurations, and two dependency-aware generation conditions: single-prompt refactoring and LangGraph-based multi-agent refactoring.
Large Language Models (LLMs) are increasingly used to refactor source files, but file-local evaluations do not reveal changes to contracts, dependencies, or caller assumptions affecting repository adoption.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends
- Oct 16, 2026CME-SpectrumBench: Can LLMs Analyze Condensed Matter Spectral Data?
- Oct 16, 2026Language Models Learn to Improve Language Models
- Oct 15, 2026Progress Reward Model for Reinforcement Learning via Large Language Models
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
- Oct 15, 2026Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration