On the Correctness of Software Merge

Abstract: Three-way merge tools play crucial roles in modern software development, where a developer forks a branch to make local modifications and requests it to be merged into the main branch via a "pull request." Despite its importance, the task has traditionally been defined in an intuitive manner, and the results of merge tools are often accepted without scrutiny. In this paper, we present a new structural merge tool in comparison with existing tools based on the syntactic criteria we propose for evaluating the merge results. We require the merge result to be both parsable and universal. Being parsable means that the result is syntactically valid according to the grammar of the programming language. Being universal means that the result incorporates all and only the edit operations occurring in each branch while ensuring that edits common to both branches are applied only once. This requirement can be precisely defined using the notion of pushouts in category theory. In a large-scale experiment involving 43,774 file merge scenarios from 76 open-source Java projects, we found a number of incorrect results reported by existing tools such as the Git companion merge tool, whereas our tool reports none. We further compared d3j's results with 2,582 developer-resolved merges and with 2,459 merge scenarios involving 21 refactoring types. These experiments revealed both the strengths and current limitations of structural merge, and underscore the importance of clear correctness criteria. We expect that the proposed criterion will provide a foundation for developing more reliable and principled merge tools.

• The paper introduces formal correctness criteria for structural software merges based on ASTs and category theory.
• It utilizes an LR(1) parser and a Zhang-Shasha tree edit distance approach to apply node-wise edits, guaranteeing syntactic validity.
• Empirical evaluation over thousands of merges shows that the d3j tool outperforms traditional line-based merge tools in accuracy and universality.

Formal Correctness in Software Merge: Structural Synthesis and Empirical Evaluation

Pull-Based Development and the Merge Problem

Pull-based software development workflows facilitate asynchronous collaboration, with contributions integrated through merges (Figure 1). Traditional three-way merge tools, such as diff3 and git-merge, operate line-wise, relying on textual sequence alignment. Their limitations—conservativeness in conflict detection and inability to handle granular code modifications—have driven the adoption of structural merge tools that operate on Abstract Syntax Trees (ASTs) and recognize semantic code edits.

Figure 1: Pull-based software development illustrating branch divergence and subsequent merge.

Structural Merge: System Architecture

The presented merge tool, d3j, adopts a strictly formal approach to merging via ASTs. Parsing, AST comparison, and AST merging are performed using custom components (Figure 2). Parsing employs an LR(1)-generated grammar, while entity matching is conducted through an approximated Zhang-Shasha tree edit distance optimized for move detection and relabeling. Edit operations—delete, insert, relabel, and move—are applied node-wise, encoded as order-preserving partial inclusion maps (Figure 3).

Figure 2: Merge system overview with AST-driven transformation pipeline.

Figure 3: Tree edit operations: delete (green), insert (blue), relabel (orange), move (gray).

Category-Theoretic Correctness Criteria

The paper introduces syntactic and universal correctness, formalized via pushouts in the category of sets and partial inclusion maps. Correctness demands:

• Parsability: the merge result must be syntactically valid.
• Universality: all and only the edits from each branch are included; common edits are applied exactly once.

Pushouts guarantee that the merge $M$ is the most general amalgamation of edits from $A$ and $B$, satisfying both commutativity and universality (Figure 4). Four explicit conditions are introduced: no extra insertion, no missed insertion, no extra deletion, no missed deletion. This framework rejects trivial merges and, through conditions encoded as order-preserving mappings (Figure 5), formally differentiates between correct and erroneous merges (Figure 6).

Figure 4: Completing commutative diagram of edits; pushout construction for $A$, $B$, and $O$.

Figure 5: Edit sequence visualized as order-preserving partial inclusion map on AST nodes.

Figure 6: Example where deleting inserted nodes breaks universality. $M$ is correct, $N$ is non-universal by missing insertion.

Conflict Rules and Syntactic Guarantees

d3j implements 32 Java-specific conflict rules across ~170 Python functions. These include relabel-relabel name clashing, overlapping deletions, insert-delete broken dependencies, insert-insert cardinality mismatches, relabel-delete name inconsistencies, and granular syntactic consistency constraints. These rules strictly enforce syntactic validity, flagging cases where merged edits would violate the target language’s grammar.

Figure 7: Commutativity between tree edit operations—verifying whether edits admit consistent AST transformations.

Empirical Evaluation: Large-Scale, Developer, and Refactoring Scenarios

Large-Scale Merge Experiment

43,774 Java file merge scenarios from 76 projects were analyzed using 8 merge tools. d3j reported zero syntactic or universality violations, outperforming all baseline tools in strict correctness (Table 1 in the paper). Universality violations were extensively characterized, with types and frequencies strongly linked to algorithmic choices (Figure 8).

Figure 8: Mutual coverage of correct conflict-free merges for d3j and other tools.

Developer-Resolved Comparison

Analyzing 2,582 developer-resolved Java merges, d3j automatically resolved 1,652 as correct CFMs, while 711 were formally incorrect—either syntactically invalid or violating universality. Developers frequently made suboptimal merges by favoring one branch or opportunistically copying edits, leading to universality violations. Semantic merge examples (Figure 9) were rare, highlighting the challenge of formal synthesis in resolution.

Figure 9: Example of semantic merge resolved by a human: switch $\rightarrow$ if refactoring.

Refactoring-Centric Benchmarking

On 2,459 merge scenarios involving structural refactorings across 21 types, d3j achieved the highest CCFM count. Tools relying on line-wise or non-semantic AST heuristics produced high counts of conflict-free merges but suffered from systematic universality violations, particularly missing insertions and discarding valid changes. d3j’s formal criteria sharply distinguished correct from merely compilable results.

Limitations, Threats, and Modularity

Manual validation and random sampling confirmed the reliability of the formal criteria. However, the inherent complexity of AST comparison, parser heterogeneity among tools, and the granularity of per-file measurement present threats to validity. Modular architectures would permit uniform evaluation and root-cause identification across components.

Theoretical and Practical Implications

Category-theoretic correctness criteria advance precision in merge synthesis, enforcing well-defined baselines for tool development. They reveal that automated tools, when strictly enforced, outperform humans regarding syntactic and universality guarantees, albeit at the cost of increased conflict reporting. The approach justifies the move from heuristic-driven merge acceptance to formally verified, structurally consistent amalgamation, particularly as codebases adopt deeper refactoring and asynchronous workflows.

d3j’s architecture is language-independent, contingent only on language-specific conflict rules and parsers, making it extensible to C/C++ and other languages. Semantic correctness remains an open frontier, with promising links to symbolic execution and change-impact analysis.

Conclusion

The research systematically defines and empirically validates formal correctness criteria for software merge, rooted in syntactic and universal properties. d3j operationalizes these criteria, demonstrating superiority over existing tools in rejecting incorrect merges. The research shifts evaluation protocols toward formal correctness, challenging prevailing practices and setting rigorous foundations for future merge tool design and modular benchmarking. The extension to semantic correctness and tool-independent conflict rules represents a logical next step.