Multi-way Merge: Concepts & Applications

• Multi-way merge is the process of combining more than two structured inputs in a single procedure, reusing inherent structure to avoid full recomputation.
• It is applied in domains such as large-scale k-NN graph construction, GPU-based sorting, distributed joins, and semantic program versioning to improve efficiency.
• The method balances performance and quality by using dynamic caches and reverse matching, with trade-offs evident in metrics like Recall@10 in graph merges.

Multi-way merge is the combination of more than two structured inputs in a single merge procedure rather than through a cascade of pairwise merges. In contemporary literature, the term denotes several closely related operations: merging multiple k-NN subgraphs into one graph, merging $K$ sorted runs in sorting algorithms and hardware networks, combining several relations in one distributed join, and integrating multiple program, model, or ontology versions under explicit correctness conditions (Zhang et al., 15 Sep 2025, Casanova et al., 2017, Kimmett et al., 2014, Mori et al., 8 Jul 2026).

1. Conceptual scope and recurring design questions

Across these settings, the inputs are already partially organized: subgraphs already encode within-subset neighbors, sorted runs already preserve local order, branch versions already share a common ancestor, and ontologies already contain structured correspondences. Multi-way merge exploits that existing structure instead of reconstructing the result from scratch. The central design questions are therefore how much of the original structure can be reused, how cross-input interactions are discovered, and whether one-shot merging is preferable to hierarchical pairwise composition.

Context	Inputs	Characteristic mechanism
Large-scale graph construction	Subgraphs $G_1,\dots,G_m$ on disjoint subsets $C_1,\dots,C_m$	Cross-match only among neighbors from different subsets
Sorting and partitioning	$K$ sorted runs or lists	Merge $K$ runs at once or compute cut indices without materializing the merge
Distributed joins	Relations or independently created sorted runs	One-round multi-way join or join over separate runs
Structured version integration	Base and diverged variants, models, or ontologies	Merge under semantic, syntactic, or partition-based constraints

A persistent misconception is that one-shot multi-way strategies strictly dominate pairwise cascades. The literature does not support that generalization. In large-scale graph construction, Two-way Merge is described as more exact and quality-preserving, whereas Multi-way Merge is favored when many subgraphs must be combined efficiently (Zhang et al., 15 Sep 2025). In MapReduce joins, the Afrati–Ullman one-round three-way join is often preferable for enumeration, but a cascade of two-way joins becomes preferable when aggregation can be integrated into an intermediate stage (Kimmett et al., 2014).

2. Multi-way merge in distributed large-scale k-NN graph construction

A particularly explicit formulation appears in graph construction for massive vector data. If the dataset is partitioned into disjoint subsets

$$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$

and each subset has a subgraph $G_i$, the goal is to construct a merged graph on

$C=\bigcup_{j=1}^m C_j$

from $\{G_1,\dots,G_m\}$. The Multi-way Merge method performs this in one single-node merge procedure rather than by $m-1$ hierarchical pairwise merges (Zhang et al., 15 Sep 2025).

The core idea is to reuse the already built within-subset structure and perform cross-matching only among neighbors from different subsets. The concatenated graph

$G_1,\dots,G_m$0

contains, for each element $G_1,\dots,G_m$1, neighbors only from the subset to which $G_1,\dots,G_m$2 originally belongs. A reverse graph $G_1,\dots,G_m$3 stores reverse neighbors. From these two structures, the algorithm builds a fixed support set $G_1,\dots,G_m$4 from the top $G_1,\dots,G_m$5 items in $G_1,\dots,G_m$6 and the top $G_1,\dots,G_m$7 items in $G_1,\dots,G_m$8. This support set is the stable part of the merge and is reused across iterations (Zhang et al., 15 Sep 2025).

Multi-way Merge augments that fixed support with three dynamic caches. The array $G_1,\dots,G_m$9 stores newly discovered neighbors from outside the original subset $C_1,\dots,C_m$0, $C_1,\dots,C_m$1 stores previously discovered outside-subset neighbors, and $C_1,\dots,C_m$2 and $C_1,\dots,C_m$3 cache reverse neighbors for newly and previously inserted neighbors. Inserted neighbors in $C_1,\dots,C_m$4 are flagged so that neighbors that already participated in Local-Join are not repeatedly resampled (Zhang et al., 15 Sep 2025).

The iterative procedure begins by sampling $C_1,\dots,C_m$5 random elements from $C_1,\dots,C_m$6 for each $C_1,\dots,C_m$7. In later iterations, it instead collects the max $C_1,\dots,C_m$8 flagged-true items from $C_1,\dots,C_m$9 into $K$0 and the max $K$1 flagged-false items into $K$2, propagates reverse-neighbor information through $K$3, and resets the sampled flags. Local-Join then computes distances for

$K$4

tries to insert $K$5 into $K$6 with flag true, and symmetrically inserts $K$7 into $K$8. Relative to Two-way Merge, the distinguishing step is that cross-matching occurs not only between $K$9 and $K$0, but also between $K$1 and $K$2, and within $K$3 itself, while excluding comparisons within the same original subset (Zhang et al., 15 Sep 2025).

3. Complexity, parallelism, and quality trade-offs in graph merge

The graph-construction literature makes the efficiency trade-off unusually explicit. Hierarchical Two-way Merge requires $K$4 pairwise merge calls when $K$5 subgraphs are combined. For Two-way Merge, each element participates in $K$6 merge levels, yielding

$K$7

Because Multi-way Merge combines all subgraphs at once, its stated cost is

$K$8

The paper concludes that Multi-way Merge is theoretically favored when $K$9, and often in practice even when $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$0, because actual sampled neighbors are usually fewer than $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$1 (Zhang et al., 15 Sep 2025).

Both merge methods are designed to be highly parallelizable. The outer loop over $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$2 is parallelized, distance calculations in Local-Join are parallelized, and the support structures and reverse caches are organized to reduce repeated work. This is significant because the paper’s broader framework targets graph construction when the data size exceeds the memory capacity of one node; it reports that a billion-scale k-NN graph can be built in approximately 17h when only three nodes are employed, and reports Recall@10 values of $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$3 on SIFT100M, $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$4 on DEEP100M, and $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$5 on SIFT1B in the distributed setting (Zhang et al., 15 Sep 2025).

The efficiency gain is accompanied by a small quality drop. Two-way Merge is reported to maintain stable graph quality as the number of subgraphs increases, whereas Multi-way Merge performs fewer cross-matchings than hierarchical Two-way Merge and is therefore slightly lower in quality. The degradation is quantified as about $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$6–$$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$7 in Recall@10 as the number of subgraphs increases, and with 64 subgraphs the merged graph quality remains comparable to NN-Descent from scratch. The final graph is obtained by a simple merge sort between the discovered cross-subset graph $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$8 and the concatenated graph $$C_1,\dots,C_m,\quad m>2,\quad C_i\cap C_j=\varnothing \ (i\neq j),$$9. For indexing graphs such as HNSW or Vamana, the merge is followed by post-processing diversification using the pruning rule

$G_i$0

because merged neighborhoods may violate the indexing-graph pruning condition (Zhang et al., 15 Sep 2025).

4. Sorted sequences: $G_i$1-way merging, partitioning, and hardware realizations

In sorting, multi-way merge reduces the number of merge rounds by combining $G_i$2 sorted runs at a time. GPU Multiway Mergesort (MMS) merges $G_i$3 runs recursively and uses a partitioning technique that splits one $G_i$4-way merge into independent warp-sized subproblems. Boundary positions are found by binary searches across the $G_i$5 lists, and each warp merges its own partition with a minBlockHeap whose nodes store $G_i$6 sorted values. The method is described as asymptotically optimal in terms of global memory accesses and completely free of shared memory bank conflicts; for certain conflict-heavy inputs it reports speedups up to $G_i$7 over MGPU and $G_i$8 over Thrust on the Gibson platform (Casanova et al., 2017).

Adaptive stable sorting uses a different multi-way strategy. Multiway Powersort generalizes Powersort from 2-way stable merges to $G_i$9-way stable merges by assigning each run boundary a $C=\bigcup_{j=1}^m C_j$0-way power in a virtual perfectly balanced $C=\bigcup_{j=1}^m C_j$1-ary tree and merging runs in increasing power order. The merge itself is implemented with a tournament tree, requiring at most

$C=\bigcup_{j=1}^m C_j$2

comparisons for $C=\bigcup_{j=1}^m C_j$3 runs of total length $C=\bigcup_{j=1}^m C_j$4. The paper proves the merge-cost bound

$C=\bigcup_{j=1}^m C_j$5

and reports that a 4-way implementation is about $C=\bigcup_{j=1}^m C_j$6–$C=\bigcup_{j=1}^m C_j$7 faster than 2-way Powersort, with merge cost about $C=\bigcup_{j=1}^m C_j$8 and cache misses about $C=\bigcup_{j=1}^m C_j$9–$\{G_1,\dots,G_m\}$0 of the corresponding 2-way methods (Gelling et al., 2022).

A related but distinct line of work studies partitioning without materializing the merge. Multi-Way Co-Ranking computes cut indices $\{G_1,\dots,G_m\}$1 for $\{G_1,\dots,G_m\}$2 sorted sequences such that $\{G_1,\dots,G_m\}$3 and the frontier satisfies

$\{G_1,\dots,G_m\}$4

Its merge-free index-space algorithm runs in

$\{G_1,\dots,G_m\}$5

time with $\{G_1,\dots,G_m\}$6 space, independent of $\{G_1,\dots,G_m\}$7, and generalizes the two-array co-ranking method that yields a perfectly load-balanced stable parallel merge in $\{G_1,\dots,G_m\}$8 time (Joshi, 27 Oct 2025, Siebert et al., 2013).

Hardware realizations show that the same idea can be encoded as fixed sorting stages. List Offset Merge Sorters (LOMS) place sorted input lists in a 2-D offset setup array and alternate column sorts and row sorts. For 2-way merge, the minimal set is 2 stages; for a List Offset 2-way sorter with 32 values per list, the paper reports $\{G_1,\dots,G_m\}$9 nS and a speedup of $m-1$0 versus a comparable Batcher device. For 3-way merge, a LOMS device merging 3 sorted input lists with 7 values each fully merges the 21 values in $m-1$1 nS, a speedup of $m-1$2 versus the comparable state-of-the-art 3-way merge device (Kent et al., 11 Jul 2025). A different hardware-network formulation merges $m-1$3 sorted lists of $m-1$4 values each in

$m-1$5

stages using $m-1$6-sorters as basic building blocks (Shi et al., 2014).

5. Distributed joins and the choice between one-shot and cascaded composition

In distributed joins, multi-way merge appears as a one-shot combination of relations that share attributes. For the three-way join

$m-1$7

the Afrati–Ullman one-round algorithm, denoted 1,3J, distributes tuples across a $m-1$8 reducer grid and performs the join in one MapReduce round. Its optimized communication cost is

$m-1$9

The cascaded two-round alternative, 2,3J, computes $G_1,\dots,G_m$00 with cost

$G_1,\dots,G_m$01

The trade-off is explicit: 1,3J avoids the materialized intermediate join and is often preferable when the goal is to enumerate the join result directly, whereas 2,3JA becomes preferable when aggregation can be pushed into the intermediate stage (Kimmett et al., 2014).

At the query-execution level, multi-join order can be represented as a binary tree traversed in reverse Polish notation. A non-recursive stack-based sort-merge method processes many-to-many multi-join queries by post-order traversal, eliminating recursion overhead and using $G_1,\dots,G_m$02 space for the stack/tree representation. The literature distinguishes sequential join sequences, or linear trees, from general join sequences, or wide bushy trees. The reported behavior is that bushy trees outperform sequential trees for larger numbers of tuples and relations, whereas sequential trees can be faster for small workloads (Asiri et al., 2022).

A different response to the merge bottleneck is to avoid the classical final merge altogether. Massively Parallel Sort-Merge (MPSM) join algorithms generate sorted runs locally and operate on those independently created runs in parallel, rather than constructing one fully sorted relation. The paper emphasizes that the final merge step is hard to parallelize and that MPSM is NUMA-affine because sorting is carried out on local memory partitions; on a 32-core machine with one TB of main memory, it scales almost linearly in the number of employed cores and outperforms the Vectorwise parallel query engine by a factor of four (Albutiu et al., 2012).

6. Structured and semantic notions of merge in software versioning

In program version control, the relevant structure is not order but behavior and syntax. One line of work defines semantic conflict-freedom for three-way program merges over a base program $G_1,\dots,G_m$03, two variants $G_1,\dots,G_m$04 and $G_1,\dots,G_m$05, and a merge candidate $G_1,\dots,G_m$06. The merge is required to preserve every observable effect introduced by either branch relative to the base and not introduce a new behavior absent from both branches. SafeMerge checks this property compositionally by combining lightweight dependence analysis for shared fragments with precise relational reasoning for edits; on 52 real-world merge scenarios from Github, it warned on 13 cases, of which manual inspection found 11 genuine semantic conflicts and 2 false positives (Sousa et al., 2018).

A stricter structural criterion is to require a conflict-free merge to be both parsable and universal. Parsability means syntactic validity with respect to the language grammar. Universality is formalized with pushouts: the merge must incorporate all and only the edit operations occurring in each branch, while applying common edits only once. In a large-scale experiment on 43,774 file merge scenarios from 76 open-source Java projects, the structured merge tool d3j reported 0 non-WP and 0 non-Univ results, whereas Git merge reported 40 non-Univ results (Mori et al., 8 Jul 2026).

Multi-version models lift this idea from a single merge to an entire version history. They encode all versions in one typed graph and support checking well-formedness for all versions without extracting each version individually, reporting all possible merge conflicts without merging all pairs of versions, and reporting all violations of well-formedness conditions that will result for merges of any two versions independent of any merge decisions. The approach is proved correct with respect to the usually employed three-way-merge semantics. Preliminary experiments report up to about $G_1,\dots,G_m$07 speedup for well-formedness checking, about $G_1,\dots,G_m$08 faster merge-conflict detection on a smaller project, and about $G_1,\dots,G_m$09 slower conflict detection on a larger project when many elements were not shared from the initial version (Barkowsky et al., 2022).

7. Merge planning, learned composition, and partition-based integration

Recent work increasingly treats multi-way merge as a planning problem over operator choice, order, and partitioning. In model merging for LLMs, SimMerge does not introduce a new merge rule; it predicts which existing operator to use, which models to combine, and in what order, using functional and structural similarity signals computed from unlabeled probes. Because merge operators are not associative, the search space for a $G_1,\dots,G_m$10-way merge is

$G_1,\dots,G_m$11

The paper reports macro-averaged auxiliary improvement of $G_1,\dots,G_m$12 at $G_1,\dots,G_m$13, $G_1,\dots,G_m$14 at $G_1,\dots,G_m$15, and $G_1,\dots,G_m$16 at $G_1,\dots,G_m$17, and shows zero-shot transfer from 7B training to 111B 3-way merges without retraining (Bolton et al., 14 Jan 2026).

Automated model merging can also be cast as multi-fidelity optimization. A framework based on SMAC, Hyperband-style resource allocation, and Bayesian optimization introduces Layer-wise Fusion Search and Depth-wise Integration Search. The search explores Task Arithmetic, TIES-Merging, SLERP, and Linear Merging across layer groups or depth blocks, supports both single-objective and multi-objective optimization, and reports that effective merges are found with less than 500 search steps for LFS. It also reports that only $G_1,\dots,G_m$18 of MATH-LFS trials and $G_1,\dots,G_m$19 of GEN-DIS-1 trials used the full budget, and that a multi-objective configuration achieved $G_1,\dots,G_m$20 average improvement over the best base model (Su et al., 6 Feb 2025).

Ontology integration exhibits an analogous shift from binary ladders to partition-based n-ary merge. CoMerger constructs an initial merge model over multiple ontologies, groups related concepts into structurally coherent blocks, merges within each block, and then combines the blocks using distributed axioms. The reported motivation is that binary merging repeatedly creates intermediate ontologies that must be reprocessed. On the evaluated datasets, n-ary merging is on average 4 times faster than balanced binary and 9 times faster than ladder binary, and for the dataset $G_1,\dots,G_m$21 with 56 ontologies it is 31 times faster than ladder binary (Babalou et al., 2020).

This broader trajectory suggests that multi-way merge is no longer only a local combining primitive. In current research, it is also a question of merge scheduling, partition formation, operator selection, and correctness criteria, with different fields emphasizing different failure modes: approximation loss in graph construction, communication blow-up in joins, bank conflicts in hardware, and semantic or structural unsoundness in version integration.