Hierarchical Analytic Merging (HAM) Overview

• Hierarchical Analytic Merging (HAM) is a design principle that employs a staged, analytic approach to combine model deltas, summaries, or GNN layers instead of relying on simple averaging.
• In LLM merging and long-document summarization, HAM uses methods such as contribution analysis and context-grounded evidence selection to enhance performance and faithfulness.
• In continual graph learning, HAM applies layer-wise ridge regression to merge parameters and resist feature drift, though its effectiveness depends on hyperparameter tuning and data support.

Hierarchical Analytic Merging (HAM) denotes a family of hierarchical, analysis-driven merging procedures rather than a single universally standardized algorithm in the current arXiv literature. One line of work treats HAM as a descriptive synonym for Hierarchical Iterative Merging (Hi-Merging), a training-free method for unifying specialized LLMs by combining model-wise pruning and scaling with layer-wise conflict analysis (Fu et al., 14 Jun 2025). A second line uses HAM to describe context-grounded hierarchical merging for long-document summarization, where recursively merged summaries are constrained by retrieved or extracted source evidence (Ou et al., 3 Feb 2025). A third line defines HAM explicitly as a layer-wise analytic merging module for continual graph learning, where Graph Neural Network linear maps are merged by ridge regression to resist feature drift (Song et al., 3 Apr 2026). This suggests that HAM is best understood as a recurring design pattern: hierarchical decomposition plus an explicit analytic criterion for deciding what to merge, preserve, or suppress.

1. Terminological status and scope

The term “Hierarchical Analytic Merging” is not uniformly canonized across subfields. In the LLM-merging literature, one source states that “Hierarchical Analytic Merging (HAM)” does not appear as a distinct algorithm and is not widely established compared to TIES, DARE, DELLA, Fisher-weighted averaging, and related methods; the same source nevertheless treats HAM as a reasonable descriptive synonym for Hi-Merging because the method is hierarchical and guided by an explicit analytic contribution metric (Fu et al., 14 Jun 2025). In long-document summarization, the same phrase is used to characterize a divide-and-conquer summarization pipeline whose merge steps are grounded by source evidence rather than by previously generated summaries alone (Ou et al., 3 Feb 2025). In continual graph learning, by contrast, HAM is the paper’s formal name for a concrete layer-wise ridge-regression merging mechanism embedded in the Analytic Drift Resister framework (Song et al., 3 Apr 2026).

Across these usages, “hierarchical” refers to staged structure rather than a single fixed architecture. In Hi-Merging, the hierarchy is model-wise followed by layer-wise processing. In long-document summarization, it is chunk-level summarization followed by recursive summary merging. In continual graph learning, it is layer-by-layer preservation of pre-activations across the GNN hierarchy. The “analytic” component likewise varies: contribution analysis in LLM merging, evidence selection in summarization, and closed-form ridge regression in graph learning.

2. Training-free LLM merging as a hierarchical analytic procedure

Hi-Merging addresses the problem of combining multiple specialized LLMs fine-tuned from the same foundation model into a single multi-task model without additional training (Fu et al., 14 Jun 2025). Let the foundation model be $M_F$ with parameters $\theta_F$, and let specialized models have parameters $\theta_k$. The method works with deltas

$\delta_k = \theta_k - \theta_F.$

Its first stage performs model-wise pruning and scaling. Top-$p$ pruning retains only the largest-magnitude fraction of each delta,

$$\tilde{\delta}_k = \operatorname{Top}_{p_k}(\delta_k),$$

and scaling tempers the retained update,

$$\hat{\delta}_k = s_k \cdot \tilde{\delta}_k,\qquad s_k \in [0,1].$$

The resulting pre-merged model is

$$\theta_{\text{pre}} = \theta_F + \sum_{k=1}^{K}\hat{\delta}_k.$$

The second stage is layer-wise and is driven by a contribution analysis. For two-model merging, the method defines deletion impact $\alpha$, addition impact $\beta$, and overall contribution

$\theta_F$0

Layer conflict for model $\theta_F$1 at layer $\theta_F$2 is then

$\theta_F$3

and aggregate conflict severity is

$\theta_F$4

Layers are processed in descending order of $\theta_F$5. Severe conflict, partial conflict, and mutual enhancement are treated differently. In severe conflict, the smaller contributor’s layer delta is discarded. In partial conflict, stricter Top-$\theta_F$6 pruning and scaling are re-applied at the layer level. In mutual enhancement, both layer deltas are retained. The final merged parameters are

$\theta_F$7

The method is training-free in the sense that it does not perform gradient-based parameter updates during merging; its cost is dominated by pruning, scaling, and validation-set evaluations for contribution analysis. The paper reports experiments on English and Chinese MCQA and open-domain QA, including MedQA, CMExam, HealthCareMagic, and cMedQA2. With Qwen2-7B-Instruct on bilingual MCQA, Hi-Merging achieved 60.1562 MedQA accuracy and 89.0700 CMExam accuracy, with a reported +18.41% average improvement and best average rank 1.0. On bilingual QA it obtained BLEU-4 35.95 and ROUGE-L 20.38 on HealthCareMagic, alongside the highest average improvement of +36.42%. The method is also reported to be sensitive to pruning and scaling hyperparameters, dependent on a calibration or validation set, and currently validated primarily for two-model merging from the same backbone.

3. Context-grounded hierarchical merging for long-document summarization

In long-document summarization, hierarchical merging is a divide-and-conquer procedure for inputs exceeding 100,000 tokens (Ou et al., 3 Feb 2025). The baseline pipeline first splits a document $\theta_F$8 into chunks of fixed size, 8K tokens in the experiments, producing first-level chunks $\theta_F$9. Each chunk is summarized as

$\theta_k$0

Higher levels recursively merge adjacent summaries up to a maximum merging context length:

$\theta_k$1

This baseline hierarchical merging, denoted HMerge, can amplify hallucinations because later summaries are produced only from earlier generated summaries.

The context-aware version corresponds to HAM in the sense of hierarchical merging with analytic grounding. An Incorporate Context module identifies source passages $\theta_k$2 to accompany each summary. Three strategies are defined. In Replace, intermediate summaries are replaced by source passages:

$\theta_k$3

In Support, summaries remain the primary gist while passages serve as evidence for correction:

$\theta_k$4

In Cite, summaries are generated with sentence-level citations to labeled passages, and passage selection uses citation frequency plus a coverage heuristic.

Relevant context can be identified by Extract, Retrieve, or Cite. Extract uses MemSum, fine-tuned for 10 epochs with learning rate $\theta_k$5 and a maximum of 20 extracted sentences per chunk. Retrieve uses BM25 over 100-token passages. Cite requires the model to append labels such as $\theta_k$6 after each summary sentence. The evaluation is conducted on Multi-LexSum and SuperSummary, using SummaC and AlignScore for input-based faithfulness, and PRisma, ROUGE, and BERTScore for reference-based quality and factuality.

The central empirical finding is that Extract-Support is the best overall variant across domains and model scales for reference-based metrics. For Llama-3.1-8B on Multi-LexSum, Extract-Support achieved ROUGE 25.8, BERTScore 63.3, and PRisma 47.5; on SuperSummary it achieved ROUGE 21.2 and PRisma 39.2. Replace variants lead on input-based metrics, such as Retrieve-Replace SummaC 47.0 on Multi-LexSum and 47.6 on SuperSummary, but often lose coverage and therefore underperform on PRisma and ROUGE. Cite variants improve AlignScore but are harder to execute reliably and underperform Extract- or Retrieve-based Support on PRisma.

4. Layer-wise analytic merging in continual graph learning

In continual graph learning, HAM is defined formally as a mechanism for merging GNN linear transformations by ridge regression in order to resist feature drift (Song et al., 3 Apr 2026). The setting is Non-Exemplar Continual Graph Learning, where previous-task graph data are inaccessible and only non-sensitive statistics are retained. Feature drift arises because cached class prototypes become misaligned when the encoder is updated on later tasks. Analytic Continual Learning avoids this by freezing the encoder, but ADR instead adapts the encoder by iterative backpropagation and then uses HAM to merge layer-wise linear maps analytically.

For a GNN layer $\theta_k$7, let $\theta_k$8 be layer-$\theta_k$9 activations on task $\delta_k = \theta_k - \theta_F.$0, let

$\delta_k = \theta_k - \theta_F.$1

and let the target pre-activation be

$\delta_k = \theta_k - \theta_F.$2

HAM forms concatenated inputs and targets over tasks,

$\delta_k = \theta_k - \theta_F.$3

and solves the ridge objective

$\delta_k = \theta_k - \theta_F.$4

The closed-form solution is

$\delta_k = \theta_k - \theta_F.$5

To implement this without storing examples, ADR keeps only the layer-wise correlation statistics

$\delta_k = \theta_k - \theta_F.$6

so that

$\delta_k = \theta_k - \theta_F.$7

The hierarchical aspect is layer-by-layer preservation of pre-activations. Because the method preserves $\delta_k = \theta_k - \theta_F.$8 in a least-squares sense for all past tasks at layer $\delta_k = \theta_k - \theta_F.$9, and because post-activations are obtained by deterministic elementwise nonlinearities, the next layer’s inputs are preserved inductively on the stored support. The paper therefore describes HAM as ensuring “absolute resistance to feature drift” relative to the retained statistics and support. After encoder merging, Analytic Classifier Reconstruction solves a second ridge-regression problem for the linear classifier,

$p$0

optionally after feature expansion through a buffer layer.

The framework stores only correlation statistics, not raw nodes, adjacency, or per-class exemplars. The paper reports that ADR matches or surpasses frozen ACL baselines on CS-CL, CoraFull-CL, and Reddit-CL in average and final accuracy while improving plasticity, and that removing HAM causes severe drift and large drops in final accuracy. It also notes limitations: rank deficiency can make solves ill-conditioned, preservation holds on the support used to form $p$1 and $p$2, and extreme class imbalance, as in Arxiv-CL, can degrade merged performance.

5. Shared principles, guarantees, and recurring limitations

Despite their different domains, the HAM variants share a common structural logic. Each begins with independently formed local objects—model deltas, chunk summaries, or task-specific GNN layers—and then merges them in a hierarchy. Each also introduces an analytic signal that is external to naive averaging. In Hi-Merging, the signal is contribution analysis built from deletion impact, addition impact, and conflict severity. In long-document summarization, it is source-grounded context selection by extraction, retrieval, or citation. In ADR, it is the closed-form solution of a ridge objective defined over accumulated cross-correlations and autocorrelations.

The guarantees and trade-offs are likewise domain-specific. Hi-Merging is training-free and explicitly prioritizes the most conflicted layers first, but it depends on validation data and assumes aligned backbones. Context-aware summarization improves faithfulness by reintroducing source evidence at each merge level, but Replace can sacrifice coverage, and Cite is instruction-sensitive. ADR yields a formal least-squares preservation statement for pre-activations on the stored support, yet that guarantee does not extend automatically to unseen regions of the distribution. This suggests that HAM is less a single theorem than a design principle: hierarchical consolidation is paired with a measurable analytic criterion that determines what information survives each merge.

6. Acronym collisions and neighboring methods

The acronym “HAM” is heavily overloaded in machine learning and should not be assumed to mean Hierarchical Analytic Merging without contextual qualification. In CLIP-based domain generalization, HAM means “Harmonizing and Merging,” a learning-then-merging framework that trains one CLIP visual encoder per source domain, enriches batches through Sample Conflict-aware Adaptive Source Enrichment, aligns update directions through Optimization Conflict-aware Parameter Alignment, and then applies Redundancy-aware Historical Model Merging (Ding et al., 11 Jun 2025). In scalable continual learning, HAM means “Hierarchical Adapters Merging,” a LoRA-based framework that groups task adapters by similarity, prunes and concatenates them within groups, and finally linearly merges the group adapters for inference (Coleman et al., 16 Sep 2025).

The acronym also predates these uses in unrelated areas. In hierarchical reinforcement learning, HAM refers to the “Hierarchy Of Abstract Machines,” a controller formalism defined by machine states, transitions, and call relations (Panov et al., 2018). In neural memory architectures, HAM refers to “Hierarchical Attentive Memory,” a full binary-tree memory that provides $p$3 access and can learn algorithms such as merging, sorting, and binary search from input-output examples (Andrychowicz et al., 2016).

Accordingly, precise disambiguation is necessary. When the phrase “Hierarchical Analytic Merging” is used in current research discourse, it most often refers either to Hi-Merging-style hierarchical conflict analysis in LLM merging (Fu et al., 14 Jun 2025), to context-grounded recursive summarization (Ou et al., 3 Feb 2025), or to ridge-based layer-wise merging in continual graph learning (Song et al., 3 Apr 2026). The term therefore functions primarily as a cross-domain descriptor for hierarchical merge procedures whose decisions are governed by explicit analytic criteria rather than by undifferentiated averaging.