ColaUntangle: LLM-Driven Commit Untangling

• ColaUntangle is an LLM-driven framework that untangles commits into atomic concern-level groups by leveraging explicit (structural) and implicit (semantic) dependencies.
• It employs a multi-agent architecture with dedicated roles for explicit dependency resolution, implicit dependency analysis, and a reviewer agent for synthesis.
• Empirical evaluations on C# and Java datasets demonstrate significant accuracy improvements over prior baselines in commit untangling quality.

ColaUntangle is an LLM-driven framework for commit untangling, the task of partitioning a tangled commit into coherent concern-level groups that better approximate atomic commits. In the formulation given by the paper, a commit is ideally atomic when it addresses one concern such as a bug fix, feature addition, or refactoring, whereas a tangled commit mixes changes for multiple concerns. ColaUntangle is presented as “the first LLM-driven collaborative consultation model” for this task, with its central design principle being the explicit separation and joint reasoning over explicit dependencies and implicit dependencies among code changes (Hou et al., 22 Jul 2025).

1. Problem setting and motivation

ColaUntangle is situated in the broader problem of recovering concern boundaries inside a single version-control commit. The paper defines a commit as the basic unit of change in version control and treats atomic commits as a best practice because they improve readability, code review, maintenance, and the quality of downstream mining tasks such as defect prediction and localization (Hou et al., 22 Jul 2025). A tangled commit instead mixes changes for multiple concerns.

The motivation is empirical as well as methodological. The paper notes evidence that roughly 11%–40% of commits can be tangled. It further attributes tangling to practical constraints: developers work under time pressure, may not clearly know concern boundaries during implementation, and often interleave bug fixing, refactoring, tests, documentation, or formatting changes. On that basis, commit untangling is framed as the task of splitting one tangled commit into several coherent concern-level groups (Hou et al., 22 Jul 2025).

A central claim of the work is that prior methods do not explicitly distinguish two qualitatively different forms of relatedness among changes. Explicit dependencies are visible structural relations such as data flow, control flow, containment, and static references. Implicit dependencies are semantic or conceptual relations such as shared intent, semantically coherent activity, highly similar edits, or cosmetic-edit coherence. This distinction is the conceptual foundation of ColaUntangle (Hou et al., 22 Jul 2025).

2. Conceptual model of dependency reasoning

The paper organizes earlier commit-untangling methods into three families: rule-based approaches, feature-based approaches, and graph-based / graph-clustering approaches (Hou et al., 22 Jul 2025). Rule-based approaches use manually designed signals such as line distance, textual similarity, def-use/use-use links, or confidence voters. Feature-based approaches classify change pairs using handcrafted features like “same method,” “same class,” and “same package.” Graph-based methods build structures such as PDGs, delta-PDGs, entity-reference graphs, or diff-hunk graphs and then cluster using agglomerative clustering, graph partitioning, or GNNs.

ColaUntangle’s criticism is not that these approaches are entirely ineffective, but that they tend to collapse heterogeneous evidence into a single signal. The paper argues that they are shallow, brittle, expensive to engineer, weak on deeper semantic intent, or overly dependent on training data or project-specific structure, and that they do not explicitly distinguish explicit from implicit dependencies (Hou et al., 22 Jul 2025).

The dependency taxonomy in ColaUntangle is qualitative rather than metric-defined. For explicit dependencies, the paper gives two rules: “Code changes with data dependencies often belong to the same concern” and “Code changes with control dependencies often belong to the same concern.” For implicit dependencies, it gives three rules: “Code changes with semantic similarity may belong to the same concern”; “Code changes with high textual or structure similarity may belong to the same concern”; and “Code changes introduced for cosmetic edits, such as syntactic formatting, refactoring, or non-functional textual modifications, often belong to the same concern” (Hou et al., 22 Jul 2025).

The examples used to motivate this distinction are representative. One example groups creating, querying, and deleting an index in a test as a semantically coherent activity even without obvious direct control or data dependence among all edited lines. Another groups an enum definition with a sorting method that uses it through data/control dependence. Further examples concern repeated replacements such as LicensesCharset to StandardCharsets, and cosmetic edits such as formatting or refactoring, both of which the authors treat as forms of implicit dependency (Hou et al., 22 Jul 2025).

3. Multi-agent architecture and collaborative consultation

ColaUntangle is implemented as a three-agent collaborative system consisting of an explicit-dependency agent (EA), an implicit-dependency agent (IA), and a reviewer / synthesis agent (RA) (Hou et al., 22 Jul 2025). The novelty lies not only in using an LLM, but in decomposing the task into dependency-specific reasoning roles and reconciling them through iterative consultation.

The explicit-dependency agent specializes in untangling based on explicit dependencies. It receives the commit diffs, denoted Info_{diffs}, together with the explicit contexts extracted from the delta-PDG, denoted Info_{EC}. It outputs an initial untangling result $r_{EA}$, explanations for its grouping decisions, and later a validation opinion on the reviewer’s synthesis, including agreement or disagreement and, when disagreeing, a revised untangling result with rationale (Hou et al., 22 Jul 2025).

The implicit-dependency agent specializes in untangling based on implicit dependencies. It receives Info_{diffs} and the implicit contexts from the delta-PDG, denoted Info_{IC}. It analogously outputs an initial untangling result $r_{IA}$, explanations, and a validation opinion on the reviewer’s synthesis (Hou et al., 22 Jul 2025).

The reviewer / synthesis agent is responsible for integrating these perspectives. Its role prompt characterizes it as an expert reviewer with expertise in both explicit and implicit dependencies. It performs three functions: synthesize, revise, and stop. Initially it receives $r_{EA}$, $r_{IA}$, the commit diffs, and both explicit and implicit contexts, and produces a synthesized untangling result $r_{RA}$ together with explanations (Hou et al., 22 Jul 2025).

The consultation loop is iterative and reviewer-centered. The paper specifies the process as follows: EA and IA first untangle the commit independently; RA synthesizes their outputs; EA and IA validate the synthesis; if both agree, the process stops; otherwise RA revises and the cycle repeats until either consensus is reached or the maximum number of rounds is exhausted. The maximum number of rounds is $t=3$ (Hou et al., 22 Jul 2025). The paper gives the following pseudocode:

$r_{EA}$7

The prompting scheme is only partially specified. The paper states that prompts include role definition, dependency rules relevant to the agent, and structured code information, but it does not provide full prompt templates, confidence scoring, explicit voting mechanisms, self-consistency sampling, temperature settings, tool calling beyond graph-derived context, or formal output schemas (Hou et al., 22 Jul 2025).

4. delta-PDG and structured context construction

A key technical component of ColaUntangle is the multi-version Program Dependency Graph, written as $\delta$-PDG (Hou et al., 22 Jul 2025). The paper first defines a standard PDG as a directed graph whose nodes $N$ represent program statements or conditional expressions and whose edges $E$ represent data flow or control flow. It then states that “The multi-version program dependency graph ($\delta$-$r_{IA}$0) is the disjoint union of all nodes and edges across all versions in the PDGs at versions $r_{IA}$1 and $r_{IA}$2.”

This graph is required because commit untangling concerns relationships among edited elements across versions, not within a single program snapshot. Rather than passing the entire graph to the LLM, ColaUntangle derives two structured views from it (Hou et al., 22 Jul 2025).

The explicit context is a compressed delta-PDG that retains only modified nodes and their dependency relationships. If two changed nodes do not have a direct edge but are connected by a dependency path, the framework adds a compressed edge between them, labeled by the variables involved in the data flow along the original path. This view is intended to preserve explicit dependency information while reducing graph size (Hou et al., 22 Jul 2025).

The implicit context retains modified nodes together with their one-hop neighbors in the delta-PDG. The purpose of this view is not to encode semantics directly as graph edges, but to expose enough surrounding code context for the LLM to infer semantic, conceptual, similarity-based, or cosmetic relations indirectly (Hou et al., 22 Jul 2025).

The graph formalism remains deliberately lightweight. The paper does not provide edge-weight equations, learned relation weights, explicit semantic edges, or a formal optimization objective over graph structure. It also does not define a numeric dependency score $r_{IA}$3, a clustering loss, or a pairwise classification objective. This suggests that the graph serves as structured symbolic context for LLM reasoning rather than as the locus of an end-to-end learned graph algorithm (Hou et al., 22 Jul 2025).

5. Workflow, evaluation protocol, and empirical results

Operationally, ColaUntangle takes a tangled commit, obtains its diffs and changed statements, constructs the delta-PDG, derives explicit and implicit contexts, invokes EA and IA for initial untangling, lets RA synthesize the two outputs, iterates consultation if needed, and returns the final reviewer output as the untangled grouping (Hou et al., 22 Jul 2025). The paper expresses the worker and reviewer stages with the following formulas:

$r_{EA}$8

and for later rounds,

$r_{EA}$9

with revision and stopping defined analogously (Hou et al., 22 Jul 2025). The paper notes minor typographical inconsistencies in these formulas.

The evaluation uses two widely used datasets: a C# dataset with 1,612 tangled commits from 9 GitHub projects, each with $r_{IA}$4 concerns, and a Java dataset with over 14k tangled commits from 10 GitHub repositories, with concerns per commit ranging from 2 to 32 (Hou et al., 22 Jul 2025). In both cases the tangled commits are synthetic: atomic commits were selected using criteria such as temporal proximity, namespace similarity, and file coupling, and then artificially tangled using git cherry-picking.

Untangling quality is measured with two cluster-assignment metrics. For changed statements, the paper defines

$r_{IA}$0

and for all statements,

$r_{IA}$1

Java prior work reported only $r_{IA}$5, and the paper follows that convention for fair comparison on Java (Hou et al., 22 Jul 2025).

The main experiments use DeepSeek-V3 via Azure OpenAI service, in zero-shot mode. Sensitivity analysis also tests GPT-4o-mini, GPT-4o, Qwen3-235b, and Claude-4-sonnet (Hou et al., 22 Jul 2025).

On the C# dataset, the overall scores reported are 72 / 93 for ColaUntangle on $r_{IA}$6. The strongest prior baseline is HD-GNN at 50 / 90, with other baselines including UTango at 46 / 90, Flexeme at 34 / 83, $r_{IA}$7-PDG+CV at 35 / 84, Herzig et al. at 28 / 66, and Barnett et al. at 7 / 13. The LLM-only baselines are $r_{IA}$8 at 60 / 90 and $r_{IA}$9 at 61 / 90 (Hou et al., 22 Jul 2025). The paper’s “44% improvement” claim on C# refers specifically to the change in $r_{EA}$0 from 50 to 72 over HD-GNN.

On the Java dataset, ColaUntangle reports 76 on $r_{EA}$1, compared with 38 for HD-GNN, 34 for UTango, 30 for SmartCommit, 24 for Base-2, 23 for Herzig et al., and 17 for both Barnett et al. and Base-1. The LLM baselines score 63 for $r_{EA}$2 and 66 for $r_{EA}$3 (Hou et al., 22 Jul 2025). The “100% improvement” claim refers to the increase from 38 to 76 over HD-GNN on $r_{EA}$4.

Ablation results reinforce the role of consultation. Table 3 reports w/o Info Tools at 67/92 overall, w/o Collaborative at 61/91, w/o explicit worker + w/o collaborative at 59/90, w/o implicit worker + w/o collaborative at 60/91, and ColaUntangle at 74/94 overall (Hou et al., 22 Jul 2025). The paper emphasizes that removing collaborative consultation causes the largest degradation, from 74 to 61 overall $r_{EA}$5, while removing information tools reduces it from 74 to 67.

The reported runtime and cost are also explicit. The average inference time is 125.13 seconds per example, and the cost is \$2.92 per 100 examples, about 2.92 cents per example (Hou et al., 22 Jul 2025). The average number of consultation rounds is 1.38; 75.50% of examples converge in one round, 11.53% in two rounds, and 13.16% reach the maximum of three rounds.

6. Interpretation, limitations, and significance

The paper attributes ColaUntangle’s gains to four factors: the separation of explicit and implicit reasoning, the structured context from delta-PDG, the collaborative consultation mechanism, and the semantic capabilities of LLMs, particularly for conceptual links, clone-like similarity, and cosmetic edits (Hou et al., 22 Jul 2025). The empirical gap over the single-LLM baselines—72 vs 60–61 on C# and 76 vs 63–66 on Java—is used to support the claim that the gains derive from the structured multi-agent framework rather than from LLM usage alone.

The qualitative analysis further supports this interpretation. The paper reports that 48.91% of commits exhibited initial disagreement between the explicit and implicit workers, which it takes as evidence that consultation is substantively necessary rather than a superficial design choice (Hou et al., 22 Jul 2025). It also notes that accuracy tends to decrease as commits contain more changed statements. The authors suggest that this may occur because LLMs lose context on larger commits or because the model splits commits more finely than the human annotations expect.

The error analysis of 50 cases identifies both over-splitting and under-merging behavior. Under fine grouping granularity, the paper reports Judgment error: 14, Composite commit: 17, and File formatting: 3. Under coarse grouping granularity, it reports Similar change: 11 and Mergeable logic: 5 (Hou et al., 22 Jul 2025). The discussion explicitly raises the possibility that some apparent errors may reflect issues in the benchmark itself, because the synthetic atomic commits used to generate the tangled commits may already contain multiple activities.

The limitations are also explicit. The paper notes that the dependency definitions may be incomplete, delta-PDG quality depends on static analysis tools, public repositories introduce a potential data leakage risk, the datasets are artificial, the model may mismatch annotation granularity, inference incurs latency and cost, evaluation covers only C# and Java, and larger commits remain difficult because of LLM reliability and context limitations (Hou et al., 22 Jul 2025). It does not claim a supervised loss-minimization formulation, a general optimization equation, or a generalizable numerical dependency model.

Within the literature on commit untangling, ColaUntangle is best understood as a framework that redefines the untangling problem as collaborative dependency reasoning over changed statements. Its main technical identity lies in coupling structured program-analysis context from $r_{EA}$6-PDG with zero-shot multi-agent LLM consultation. A plausible implication is that the framework’s contribution is as much epistemic as algorithmic: it imposes an interpretable decomposition of evidence into structural and semantic channels, then requires an explicit synthesis step to reconcile them. Under that interpretation, ColaUntangle represents a shift from monolithic clustering toward reviewer-mediated, explanation-producing untangling (Hou et al., 22 Jul 2025).