Distributed Hierarchical Contour Trees

• Distributed Hierarchical Contour Trees are a distributed representation of contour trees that balance local and global data to enable scalable topological queries on large scientific datasets.
• They leverage efficient fan-in/fan-out strategies, hypersweeps, and branch decomposition to compute precise geometric properties and subtree measures.
• The approach integrates lazy attachment augmentation and pointer-doubling for accurate contour extraction, achieving near-ideal scaling even with complex datasets.

Searching arXiv for the specified paper and closely related contour-tree work to ground the article in published sources. Distributed hierarchical contour trees (DHCTs) are distributed representations of contour trees for large-scale scientific data, intended for distributed computation and storage in high-performance computing settings. In the formulation summarized in "Distributed Augmentation, Hypersweeps, and Branch Decomposition of Contour Trees for Scientific Exploration" (Li et al., 2024), a DHCT is not only a storage scheme for the global contour tree but also a substrate for augmentation, hypersweeps, branch decomposition, and contour extraction. The central problem addressed is that, although recent work introduced distributed hierarchical contour trees for distributed computation and storage of contour trees, effective use of these distributed structures in analysis and visualization requires subsequent computation of geometric properties and branch decomposition to support contour extraction and exploration (Li et al., 2024).

1. Distributed representation and hierarchy

Let $M$ be a $d$-dimensional mesh of $N$ cells and $N$ vertices. The global contour tree is written as $T=(V,A)$, where $V$ is the vertex set and $A$ is the arc set. The representation used for DHCTs distributes $T$ across $P$ MPI ranks by assigning each rank $r$ a contiguous block of the mesh $d$0, with blocks forming a decomposition $d$1 and overlap on block boundaries (Li et al., 2024).

On each rank, the stored object is a local sub-tree $d$2 such that

$d$3

and

$d$4

Across all ranks, $d$5 and $d$6, so the global contour tree is recovered as the union of the local sub-trees, with consistent global identifiers. A process-assignment function

$d$7

specifies ownership, with $d$8 when $d$9 is stored on rank $N$0 as part of $N$1's local sub-tree. On shared arcs or shared nodes, the descriptor is replicated on each rank in the union.

The hierarchy arises from a recursive fan-in of merge operations over $N$2 rounds, followed by a fan-out reinsertion of suppressed interior branches. Rounds are labeled $N$3. At level $N$4, ranks are paired or grouped to exchange and merge contour-tree fragments. In the final DHCT, each vertex $N$5 stores an integer $N$6 indicating the level at which it was introduced or reinserted during the fan-out. The vertex set is further decomposed as $N$7, where $N$8 is the set of critical points and $N$9 the set of inserted regular nodes used to subdivide arcs for correct prefix-sum ordering; each arc $N$0 is annotated by the level $N$1 at which it was last modified.

This representation makes the contour tree simultaneously global and rank-local. A plausible implication is that the DHCT hierarchy is designed to preserve enough structural redundancy for parallel topological queries, while still aligning the topology with a mesh decomposition natural to MPI-style execution.

2. Augmentation of lazy attachment points

After the DHCT has been constructed through the fan-in/fan-out procedure of Carr et al., some regular attachment points remain lazy: they are recorded only on the rank owning the child subtree and are not inserted on the parent arc across all ranks. The augmentation algorithm inserts each such point as an explicit node along its parent superarc on every rank that holds that superarc (Li et al., 2024).

The algorithm has two phases. In the fan-in exchanges, for levels $N$2, all attachment points on arcs introduced at level $N$3 are identified and exchanged between the two partner ranks of that round. After $N$4 rounds, every rank holding a given arc knows all of its to-be-inserted attachment points. In the fan-out insertion, for levels $N$5, the local DHCT is reconstructed by splitting each arc $N$6 at level $N$7 into a sequence of arcs through the sorted list of new attachment points.

The exchanged data are stored as attachment-point lists $N$8, where $N$9 is a list of $T=(V,A)$0 for each lazy attachment on arcs with level $T=(V,A)$1 that originate from rank $T=(V,A)$2's local suppressed branches. At each level, a partner is defined by $T=(V,A)$3; lists are exchanged with MPI_Sendrecv, merged, and deduplicated. In fan-out insertion, each surviving arc $T=(V,A)$4 is replaced by a chain of arcs through the attachment points sorted by $T=(V,A)$5-value, after which vertices in the new local DHCT are renumbered consecutively.

Communication cost per level is $T=(V,A)$6 words. Because each attachment originates from a boundary between blocks, the typical three-dimensional case gives $T=(V,A)$7 per round. Over $T=(V,A)$8 rounds, the communication volume is

$T=(V,A)$9

Local work per level is dominated by merging and sorting $V$0, at cost $V$1. The report notes that, in the worst case, one could have $V$2 attachments, giving $V$3 total work, but that in practice $V$4 and attachment-point lists remain small.

A common misconception is that distributed storage of contour trees is by itself sufficient for downstream scientific exploration. The augmentation procedure directly contradicts that view: distributed prefix operations, called hypersweeps, require those lazy attachment points to be inserted explicitly on every relevant rank.

3. Hypersweeps and distributed subtree measures

Once the DHCT is fully augmented, every rank has, for each hyperarc $V$5, the exact ordered list of its supernodes. A hyperarc is defined here as a maximal monotone chain of superarcs directed toward the global root. This makes it possible to compute any associative subtree measure by a distributed prefix-sum, termed a hypersweep, plus a fan-in reduction (Li et al., 2024).

The procedure has three steps. First, each rank computes a per-arc measure $V$6 on each arc $V$7. The explicit example given is

$V$8

the number of regular mesh vertices in the topological zone $V$9. Second, per-arc measures are converted to per-hyperarc prefix sums

$A$0

by an $A$1 shared-memory parallel scan across the $A$2 arcs in $A$3. Third, a distributed fan-in reduction accumulates each hyperarc's final local value across ranks that hold pieces of $A$4, yielding the global subtree measure

$A$5

For a hyperarc with ordered sequence $A$6, the local subtree contribution on rank $A$7 is

$A$8

The global quantity is then computed as

$A$9

via MPI_Allreduce (sum). For boundary arcs, designated as ghost arcs, each mesh vertex is assigned exclusively to the rank of highest block-ID that holds it, specifically to avoid double-counting.

The complexity summary is explicit. Local prefix scans cost $T$0 time. MPI_Allreduce on $T$1 hyperarcs costs $T$2 latency plus $T$3 words. Overall time per hypersweep is approximately

$T$4

This formulation situates the DHCT not merely as a topological skeleton but as a distributed query structure for subtree volumes and related geometric properties. A plausible implication is that the ordered hyperarc representation is the critical bridge between topological connectivity and scalable prefix-style aggregation.

4. Branch decomposition and significance measures

The branch decomposition is computed in three stages: assigning each supernode its best ascending and descending continuation, exchanging those choices to ensure global consistency, and applying pointer-doubling to identify each monotone branch set (Li et al., 2024).

In the first stage, each supernode $T$5 examines its up-adjacent arcs $T$6 and down-adjacent arcs $T$7 and selects the one whose significance $T$8 is maximal. The report states that significance may be arc-length, estimated volume, or persistence. In the second stage, over the same $T$9 rounds used for augmentation fan-in, partner ranks exchange, for each shared supernode, $P$0 and $P$1. Upon receipt, each rank updates

$P$2

and similarly for $P$3. After $P$4 rounds, all ranks agree on these choices for every shared supernode.

In the third stage, each chosen best-up pointer is oriented toward the root of the hyperstructure, and pointer-doubling, described as parallel tree-contraction, is used to find the root representative $P$5 for each vertex in $P$6 steps. The set

$P$7

forms a branch $P$8 rooted at $P$9.

The significance metrics are defined directly from arc and hyperarc quantities. For an arc $r$0 with end isovalues $r$1,

$r$2

where $r$3 is the subtree volume obtained from the hypersweep on hyperarc $r$4. The persistence of a branch $r$5 is

$r$6

and its volume can be written as

$r$7

The report states that $r$8 is typically used as $r$9. After pointer-doubling, a local reduction by root-ID computes branch volumes or persistences through MPI_Allreduce by key (r_v) sum (\phi(a)).

This decomposition supplies the ranking mechanism needed for later contour queries. It also clarifies that branch extraction is not presented as an independent geometric post-process; instead, it is integrated with the distributed topological structure through shared continuation choices and global reductions.

5. Query structure and contour extraction

After branch significances $d$00 have been computed, the DHCT is used as a query structure for selecting important contours. The selection step chooses the top-$d$01 branches $d$02, and one representative contour per branch is extracted by choosing an isovalue at the branch's saddle-end:

$d$03

where $d$04 is the lower, saddle end of $d$05, and $d$06 is an infinitesimal value above $d$07. In practice, the implementation uses the next-higher floating-point value (Li et al., 2024).

The distributed query proceeds in four phases. First, top-$d$08 branches are selected globally by repeated partner exchanges over levels $d$09, merging local top-$d$10 lists. Second, for each selected branch, the corresponding saddle supernode $d$11 is found and $d$12 is set to nextfloat(f(v_s)). Third, each rank extracts local contours by iterating over each cell $d$13 in $d$14 and each selected branch $d$15, testing whether $d$16. When the test passes, MarchingCells computes a fragment $d$17. The local hyperarc $d$18 for that fragment is identified, and its branch identifier $d$19 is obtained through hyperstructure lookup; the fragment is appended to local_mesh[B] only if $$B_h = B<code>. Fourth, the per-branch meshes may optionally be gathered or rendered, with</code>MPI_Gatherv` used if global mesh assembly is desired.</p> <p>The communication and local-work characteristics are narrowly specified. In step 1, communication is$$d$20 words. Step 3 requires no communication because cells remain local. Final gathering is optional. Local work consists of $d$21 cell tests plus $d$22 hyperstructure lookups each.

This query model emphasizes that the branch decomposition is operational rather than purely descriptive. A common misunderstanding would be to treat topological branches as only abstract summaries; here, they are used to drive explicit contour extraction in distributed memory.

6. Performance characteristics and scaling behavior

The reported implementation was evaluated on NERSC Perlmutter using AMD EPYC 7763, $d$23 cores per node, InfiniBand, up to 512 nodes, 32 768 MPI ranks, and 128 OpenMP threads per node (Li et al., 2024). The four datasets tested were GTOPO, WarpX, Nyx, and MICrONS.

Dataset	Single-node time (s)	Speed-up
GTOPO	85.4	17.2×
WarpX	102.1	98.8×
Nyx	519.3	7.5×
MICrONS	—	6.8×

The dataset descriptions and headline scaling results are given explicitly. GTOPO is a 2D dataset of size $d$24, with speed-up approximately $d$25 at 512 nodes over single-node PPP. WarpX is a 3D dataset of size $d$26, with speed-up approximately $d$27 at 256 nodes. Nyx is a 3D dataset of size $d$28, with speed-up approximately $d$29 at 512 nodes. MICrONS is 3D $d$30 EM data and exhibits scaling similar to Nyx.

The phase-by-phase analysis distinguishes between topological regimes. In strong scaling at fixed problem size, phase (1), DHCT fan-in/fan-out, is dominated by local SMP merge-tree cost and scales well. Phases (4), augmentation, (5), hypersweep, and (6), branch decomposition, drop roughly by one half when doubling nodes for clean topology datasets such as WarpX and GTOPO. For noisy topology datasets such as Nyx and MICrONS, communication of boundary and hyperarc data becomes the bottleneck, flattening strong scaling beyond approximately 128 nodes. Phase (7), contour extraction, scales almost perfectly because only $d$31 exchange is required for small $d$32, with $d$33 given as an example.

The hardware configuration is specified as Perlmutter CPU-only nodes with 64 cores per socket, two sockets, 512 GB RAM, EDR InfiniBand, Cray MPICH for MPI, and OpenMP threading. The summary conclusion is that the distributed DHCT algorithms for augmentation, hypersweeps, branch decomposition, and contour extraction achieve near-ideal scaling on data with well-behaved topology and remain feasible, though communication-bound, for highly complex datasets, enabling interactive-scale contour queries on meshes far larger than any single node's memory (Li et al., 2024).

A plausible implication is that the primary scalability constraint is not the contour-tree abstraction itself but the interaction between topological complexity and boundary-induced communication. That interpretation is consistent with the observed divergence between clean-topology and noisy-topology datasets.