---
title: Hierarchical Node Compression (HNC)
url: https://www.emergentmind.com/topics/hierarchical-node-compression-hnc
type: topic
---

# Hierarchical Node Compression (HNC)

Hierarchical Node Compression (HNC) is a data augmentation strategy for tree-structured reasoning processes, developed to enhance the stability, diversity, and robustness of reward models in large language model (LLM) training. By compressing segments of Monte Carlo Tree Search (MCTS)-generated reasoning trees through controlled merging of parent–child nodes, HNC amplifies the variety of reasoning step sequences, injects controlled label noise, and incurs minimal computational overhead in large-scale automated reasoning pipelines [2503.13551].

## 1. Motivation and Conceptual Foundations

The development of HNC originates from the need to efficiently create high-diversity, robust reasoning data for training Process Reward Models (PRMs) and, more generally, for Hierarchical Reward Models (HRMs) in LLMs. MCTS is commonly used to annotate reasoning trajectories by recursively simulating possible step sequences; however, the computational cost becomes prohibitive with deep, wide trees required for stable MC-Score estimation, with a reported budget of ∼2,457 A100-GPU hours for standard datasets. This computational bottleneck restricts tree expansion and limits the diversity of reasoning patterns present in the data.

HNC addresses this limitation by randomly merging consecutive reasoning steps within the existing MCTS trees, thus producing compressed trees that yield both finer- and coarser-grained reasoning examples. This augmentation is computationally lightweight (∼30 minutes on a single A100 GPU), enables broader coverage of reasoning sequence types, and introduces mild stochastic perturbations that are empirically shown to enhance model generalization and stability [2503.13551].

## 2. Formal Definition and Mathematical Operations

Let $T = (V, E)$ denote an MCTS tree, where each node $v \in V$ represents a partial chain-of-thought step, and edges $E$ define parent–child relationships. Each node $v$ is associated with a text descriptor $\mathrm{text}(v)$ and an MC-Score $S(v)$, defined as the normalized count of correct leaves underneath $v$:
\[
S(v) = \frac{\#\,\{\text{correct leaves under }v\}}{\#\,\{\text{all leaves under }v\}}
\]
HNC selects a subset of eligible edges $(u \rightarrow v)$ and constructs a new node $w$ via:
- Text merging: 
\[
\mathrm{text}(w) = \mathrm{text}(u)\;\|\;\mathrm{text}(v)
\]
- Score transfer: $S(w) = S(v)$.
- Tree rewiring:
  - $w$ replaces $u$’s link from its parent, if any
  - $w$ inherits $v$’s children as its own

The resulting tree $T' = (V', E')$ contains new compressed nodes, with the overall depth and step granularity reduced along merged branches. Only a controlled fraction of parent–child pairs are merged to avoid collapsing the entire structure, thus preserving essential tree diversity.

## 3. Algorithmic Implementation

The canonical algorithm for HNC is as follows:

```python
def HNC_Augment(T, p_merge):
    # Inputs: T = (V, E) original tree, p_merge ∈ (0,1)
    V_prime, E_prime = V.copy(), E.copy()
    shuffled_edges = random.shuffle(list(E))
    for (u, v) in shuffled_edges:
        if random.uniform(0, 1) < p_merge:
            w = Node()
            w.text = u.text + v.text
            w.score = v.score
            # Redirect parent 
            if exists p with (p, u) ∈ E_prime:
                E_prime.remove((p, u))
                E_prime.add((p, w))
            # Redirect children
            for c in v.children:
                E_prime.add((w, c))
            V_prime.remove(u)
            V_prime.remove(v)
            E_prime = {(x, y) for (x, y) in E_prime if x != u and x != v and y != u and y != v}
            V_prime.add(w)
    return (V_prime, E_prime)
```
In practice, $p_{\text{merge}}$ is selected to produce sufficient variability without excessive structural collapse.

## 4. Computational Complexity and Efficiency

The cost of generating the initial MCTS trees scales exponentially with tree depth and branching factor. In contrast, HNC’s augmentation pass iterates over each edge once, with only $O(1)$ operations per edge (merging, redirection, deletion/insertion). Thus, the overall complexity is $O(|E|)$, with trivial GPU/memory consumption relative to MCTS tree expansion. The time footprint is empirically stated as $\sim$30 minutes on a single A100 (80GB), compared to thousands of GPU-hours needed by MCTS for a single dataset [2503.13551].

## 5. A Step-by-Step Example of HNC on an MCTS Tree

Consider an MCTS mini-tree with the following structure:

```
Root
  ├─ Step 1: “Compute 1+2=3” (score 0.8)
  │     ├─ Step 2a: “Then 3+4=7” (score 0.5)
  │     ├─ Step 2b: “Then 3+3=6” (score 1.0)
  │     └─ Step 2c: “Then 3+5=8” (score 0.2)
  ├─ Step 1b: ...
  └─ Step 1c: ...
```

By merging Step 1 → Step 2b, the tree under that branch becomes:
```
Root
  ├─ Merged Step 1–2: “Compute 1+2=3. Then 3+3=6” (score 1.0)
  ├─ Step 1a (unchanged)
  └─ Step 1c (unchanged)
```
The merged branch becomes one level shallower, while the MC-Score is inherited from Step 2b.

## 6. Integration into the Hierarchical Reward Model (HRM) Training Pipeline

The HNC-augmented trees play a dual role in HRM pipeline construction:
1. Large sets of raw MCTS trees $T_i$ are generated per reasoning task.
2. PRMs are first trained on basic stepwise (fine-grained) pairs from these trees.
3. HNC is then applied to each $T_i$ to generate compressed variants $T'_i$ containing parent–child merges.
4. Both fine-grained (original) and coarse-grained (merged) pairs are extracted for HRM training, with labels derived from PRM or MC-Scores.
5. HRM is ultimately trained on this union, improving its ability to evaluate both individual and multi-step reasoning coherence.

This strategy ensures exposure to both diversity in short steps and robustness to variable step granularity, reflecting real-world reasoning trajectories [2503.13551].

## 7. Empirical Evaluation and Measured Impact

Empirical results reported on PRM800K, GSM8K, and MATH500 datasets indicate that HRMs trained with HNC augmentation outperform baseline PRMs in both absolute accuracy and output stability. Using the Qwen2.5-7B-Math-Instruct policy under Best-of-$N$ sampling, HRMs achieve a score of 0.655 at $N=8$ versus PRM’s 0.600, with improved stability for all $N$ up to 64. 
- Generalization to new reasoning domains (e.g., GSM8K, Math500) is also strengthened, particularly for more challenging splits.
- The augmentation process itself is negligible in computational demands, requiring less than 1/100th the resources of raw MCTS expansion.

A plausible implication is that HNC facilitates more robust training by increasing data diversity and introducing mild label noise, which improves tolerance to overfitting and enhances model performance under distributional shift [2503.13551].

---

**Summary Table: Key Operational Dimensions of HNC**

| Aspect               | Requirement/Result                                | Reference                |
|----------------------|--------------------------------------------------|--------------------------|
| Core operation       | Merge parent–child nodes in MCTS tree            | [2503.13551]             |
| Computational cost   | $O(|E|)$; minutes on single GPU                  | [2503.13551]             |
| Training impact      | +5.5 points Best-of-8 accuracy (PRM800K dataset) | [2503.13551]             |
| Usage context        | Augmentation for Hierarchical Reward Models       | [2503.13551]             |

Hierarchical Node Compression demonstrably improves the data efficiency and reliability of reward modeling for LLM-driven reasoning, with practical benefits in both empirical stability and generalization.

Source: https://www.emergentmind.com/topics/hierarchical-node-compression-hnc