Parallel Hierarchical Pooling (PHPool)

• PHPool is a multiscale pooling method that constructs a single global dendrogram to generate parallel multi-resolution cluster assignments, thereby reducing cumulative information loss.
• By leveraging hypergraph modularity and node-degree preserving projections, PHPool maintains higher-order relational structures while minimizing common topological artifacts.
• Applied in Hypergraph U-Nets and temporal action recognition, PHPool improves reconstruction, classification, and anomaly detection through efficient, globally optimized pooling.

Parallel Hierarchical Pooling (PHPool) denotes a multiscale pooling construction in which a hierarchy is built once and pooling operators at multiple resolutions are obtained in parallel rather than through a layer-by-layer sequential assignment procedure. In the hypergraph setting, PHPool was introduced together with its inverse PHUnpool in “Beyond Convolution: Advancing Hypergraph Neural Networks with Hypergraph U-Nets,” where the operators are constructed by cutting a hierarchical clustering dendrogram at different granularities and are used to retain maximal structural information from the input hypergraph (Wang et al., 8 Jun 2026). A conceptually related design appears earlier in “Deep hierarchical pooling design for cross-granularity action recognition,” which proposes a tree-structured set of temporal pooling branches computed in parallel at multiple granularities and combined by learned nonnegative, simplex-constrained weights, although the paper does not use the name PHPool (Mazari et al., 2020).

1. Conceptual basis and motivation

PHPool addresses a general problem of multiscale representation learning: pooling must reduce resolution without destroying the structural regularities that make the input meaningful. In hypergraphs, this problem is acute because hyperedges encode higher-order relations among sets of nodes, so naive reduction can break multi-way connectivity and introduce topological artifacts such as isolated hyperedges or spurious connectivity after reduction (Wang et al., 8 Jun 2026).

The 2026 formulation contrasts PHPool with two classes of prior pooling strategies. Node-drop methods such as TopKPool and SAGPool discard nodes iteratively, which can quickly destroy local neighborhood integrity and produce sparse or fragmented structure when stacking multiple layers. Cluster-labeled methods learned sequentially, such as DiffPool and MinCutPool, optimize one pooling step at a time; local optima at early layers distort the hierarchy and accumulate information loss as depth increases. PHPool is designed as a global and parallel alternative: it constructs a single hierarchical clustering dendrogram of the input hypergraph and then selects cluster assignments for all scales at once using hypergraph modularity as a global criterion (Wang et al., 8 Jun 2026).

A closely related multigranular rationale appears in the 2020 temporal action-recognition setting. There, the challenge is that action categories exhibit different temporal granularities: global pooling is too invariant and may dilute fine-grained cues, while highly localized pooling is too sensitive to misalignment and noise. The proposed coarse-to-fine hierarchy aggregates features over long intervals at upper levels and over shorter intervals at lower levels, with all nodes across all levels computed in parallel and then combined by learned weights (Mazari et al., 2020). This suggests a broader interpretation of PHPool as a design pattern for parallel multiscale aggregation, even though the exact term was introduced later for hypergraphs.

2. Hypergraph formulation underlying PHPool

In the hypergraph formulation, the input hypergraph is $G = (V, E)$, with $|V| = n$, $|E| = m$, and hyperedges $e \subseteq V$. The incidence matrix is $H \in \{0,1\}^{n \times m}$, where $H_{ve} = 1$ if $v \in e$, and $0$ otherwise. Node and hyperedge degree matrices are

$$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$

and

$$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$

Hyperedge weights are collected in $|V| = n$0, often with $|V| = n$1 in the unweighted case (Wang et al., 8 Jun 2026).

The normalized hypergraph Laplacian used in the paper follows the Zhou (2006) form,

$|V| = n$2

For modularity evaluation and shortest-path computation, PHPool uses the node-degree-preserving 2-section adjacency

$|V| = n$3

which preserves node degrees when projecting the hypergraph to a pairwise adjacency (Wang et al., 8 Jun 2026).

The clustering quality criterion is hypergraph modularity. For a hard clustering $|V| = n$4, with node label $|V| = n$5 for node $|V| = n$6, the null-model term is

$|V| = n$7

and the modularity is

$|V| = n$8

This objective supplies the criterion by which the dendrogram cuts used for PHPool are selected (Wang et al., 8 Jun 2026).

3. Construction of PHPool from a dendrogram

PHPool begins by converting the hypergraph into a metric space on nodes. The method computes all-pairs shortest-path distances on the 2-section graph induced by $|V| = n$9, with weights taken from $|E| = m$0 or set to $|E| = m$1 for unweighted cases. Distances are computed via Dijkstra or an equivalent algorithm. Agglomerative hierarchical clustering is then run on the distance matrix, using a nearest-neighbor chain implementation with $|E| = m$2 complexity, to obtain a dendrogram $|E| = m$3 with merge heights (Wang et al., 8 Jun 2026).

The central parallel step is the simultaneous selection of multiple pooling levels from that single hierarchy. The method enumerates the unique cut thresholds $|E| = m$4 at the merge heights, cuts the dendrogram to obtain partitions $|E| = m$5, and evaluates $|E| = m$6 for every unique threshold. It then selects $|E| = m$7 levels greedily with a rightward search: first the threshold $|E| = m$8 with maximum modularity is chosen, then the search moves rightward to coarser thresholds only, repeating the maximum-modularity selection until $|E| = m$9 assignments are obtained (Wang et al., 8 Jun 2026).

The resulting assignments are represented by hard clustering matrices

$e \subseteq V$0

each with exactly one $e \subseteq V$1 per row, where $e \subseteq V$2 and $e \subseteq V$3 if the top layer collapses to a supernode. These matrices are the pooling operators at the successive resolutions selected from the global hierarchy (Wang et al., 8 Jun 2026).

Feature pooling is defined by sum aggregation as

$e \subseteq V$4

An optional mean-pooling variant is

$e \subseteq V$5

with

$e \subseteq V$6

Structural pooling proceeds in two stages. First, the pairwise adjacency is coarsened: $e \subseteq V$7 Second, hyperedges are reconstructed from the coarsened adjacency by clique enumeration: $e \subseteq V$8 The paper emphasizes that hyperedges are not necessarily fewer after pooling; they are recomputed from cliques at the coarser resolution, which helps capture emergent higher-order relations among clusters (Wang et al., 8 Jun 2026).

4. PHUnpool and integration into Hypergraph U-Nets

PHUnpool is the inverse operator paired with PHPool. Given a pooling matrix $e \subseteq V$9 and coarsened features, unpooling is defined as a fixed linear expansion,

$H \in \{0,1\}^{n \times m}$0

or, with average redistribution,

$H \in \{0,1\}^{n \times m}$1

For hard $H \in \{0,1\}^{n \times m}$2 assignment matrices with one-hot rows, $H \in \{0,1\}^{n \times m}$3, so a Moore–Penrose pseudo-inverse is

$H \in \{0,1\}^{n \times m}$4

Under the average-redistribution form, the paper states that the unpooling yields an exact left-inverse on the subspace of cluster-constant signals: perfect reconstruction holds whenever $H \in \{0,1\}^{n \times m}$5 is piecewise constant on the selected clusters, and more generally the reconstruction equals the orthogonal projection onto that subspace (Wang et al., 8 Jun 2026).

Structural unpooling mirrors the feature operation: $H \in \{0,1\}^{n \times m}$6 Exact structural recovery is not guaranteed in general because coarsening is many-to-one, but the feature-space projection property gives a precise reconstruction statement for cluster-constant signals (Wang et al., 8 Jun 2026).

These operators are embedded in a Hypergraph U-Net. The convolutional block can be any hypergraph convolution; the paper gives the standard spectral/message-passing form

$H \in \{0,1\}^{n \times m}$7

and also proposes HGXConv as the default in HGUN. The encoder repeats $H \in \{0,1\}^{n \times m}$8 blocks of $H \in \{0,1\}^{n \times m}$9, applying the precomputed $H_{ve} = 1$0 at each level. The bottleneck is the final coarsest representation, often a single supernode embedding. The decoder mirrors the encoder through PHUnpool followed by convolution, with skip connections that concatenate or sum encoder features at matching resolutions. A key implementation property is that all $H_{ve} = 1$1 are computed once from the original hypergraph’s dendrogram and reused for every epoch and pass, improving efficiency, consistency, and stability by avoiding gradient flow through the discrete clustering stage (Wang et al., 8 Jun 2026).

5. Theoretical properties, complexity, and empirical behavior

The paper states an information-theoretic advantage for the parallel design. Under matched first-level coarsening, a proposition shows that

$H_{ve} = 1$2

because the parallel construction obtains the coarse representation directly from $H_{ve} = 1$3, whereas the sequential pipeline passes through an intermediate information bottleneck $H_{ve} = 1$4. This is presented as a formalization of reduced cumulative information loss in the parallel hierarchy (Wang et al., 8 Jun 2026).

PHPool and PHUnpool are also described as permutation equivariant. The dendrogram and its cuts permute consistently with node permutations; the assignment matrices $H_{ve} = 1$5 change accordingly; pooling and unpooling are linear in $H_{ve} = 1$6 and $H_{ve} = 1$7; and the convolution operators used in the model are permutation equivariant or invariant on incident sets (Wang et al., 8 Jun 2026).

The computational profile divides into one-time preprocessing and per-epoch application. Preprocessing requires $H_{ve} = 1$8 for dendrogram construction, $H_{ve} = 1$9 for modularity scoring and greedy selection over $v \in e$0 unique thresholds, and $v \in e$1 for parent–child mapping and clique bookkeeping. Each forward application of pooling or unpooling costs $v \in e$2 for features and $v \in e$3 arithmetic for the sparse operation $v \in e$4, plus clique enumeration on the reduced graph. Because $v \in e$5 shrinks with depth, total runtime per epoch is reduced relative to flat convolutions (Wang et al., 8 Jun 2026).

Empirically, the method is evaluated on hypergraph reconstruction simulation, hypergraph classification, and node-level anomaly detection. In hypergraph reconstruction on synthetic ring, grid, pyramid, and community structures, HGUN with PHPool reconstructs node coordinates with the lowest distortion; node-drop TopKPool fails on community structure and distorts boundaries, while MinCutPool is better but still shows boundary and inter-cluster artifacts. In hypergraph classification on IMDB-B, IMDB-M, COLLAB, MUTAG, PROTEINS, D&D, and NCI1, PHPool+HGXConv ranks first on $v \in e$6 datasets; on MUTAG and PROTEINS, PHPool+HGAT slightly wins, while PHPool+HGXConv is a close second. In node anomaly detection on Yelp and Amazon, HGUN achieves the best precision, F1-macro, and G-Mean among all baselines. The paper also reports per-epoch speedups of approximately $v \in e$7 on Yelp and $v \in e$8 on Amazon over a flat HGXConv with matched depth, attributing the gains to progressive coarsening. Best accuracy typically occurs at $v \in e$9–$0$0 levels, while deeper stacks may over-compress; PHPool-based HGUN degrades more gracefully than node-drop pooling under increased depth (Wang et al., 8 Jun 2026).

6. Relation to earlier parallel hierarchical pooling in temporal action recognition

The term “Parallel Hierarchical Pooling (PHPool)” is not used in “Deep hierarchical pooling design for cross-granularity action recognition,” but the paper introduces what it explicitly describes as a parallel, hierarchical pooling or aggregation scheme: a tree-structured set of temporal pooling branches computed in parallel at multiple granularities and combined by learned nonnegative, simplex-constrained weights (Mazari et al., 2020).

For a video with $0$1 frames and hierarchy depth $0$2, level $0$3 partitions the temporal domain into $0$4 equal sub-intervals, and each node $0$5 averages frame features within its interval. With per-frame features $0$6, the node descriptor is

$0$7

At $0$8, pooling spans the whole video and is maximally invariant but least localized; as $0$9 increases, interval lengths shrink by a factor of $$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$0, producing more localized and less invariant descriptors. Because the pooling is normalized by interval length, each node representation is video-length agnostic (Mazari et al., 2020).

The paper defines two global hierarchical pooling variants. In weighted concatenation,

$$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$1

with kernel

$$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$2

In weighted averaging,

$$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$3

with kernel

$$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$4

The cross-terms in the averaging kernel relax alignment between corresponding temporal nodes and increase robustness to mis-trimming and temporal misalignment. The weights satisfy $$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$5 and $$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$6, and are learned either through a joint max-margin optimization with one-vs-rest SVMs or through a decoupled “Deep MKL” contrastive objective with softmax-reparameterized simplex constraints (Mazari et al., 2020).

The 2020 experiments on UCF-101 split-2 report best fused performance up to $$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$7 for averaging at $$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$8, compared with a Two-stream ResNet baseline at $$D_v = \mathrm{diag}(d(v_1), \dots, d(v_n)), \qquad d(v) = \sum_e H_{ve},$$9, while global average pooling at the root level only is reported at $$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$0 for appearance. Averaging increases steadily with depth, whereas concatenation peaks at $$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$1 and drops at $$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$2, which the paper attributes to greater sensitivity to misalignment in the most localized nodes (Mazari et al., 2020).

This earlier temporal model and the later hypergraph PHPool share the same central architectural idea: all branches or levels are evaluated in parallel within a single hierarchy, and multiscale information is merged by a globally defined mechanism rather than by a sequence of locally optimized pooling steps. A common misconception is therefore to treat the acronym “PHPool” as if it had originated in the 2020 work; the record in the papers on arXiv is narrower. The 2020 study presents a method that is conceptually parallel and hierarchical, while the explicit terms PHPool and PHUnpool are introduced in the 2026 hypergraph U-Net formulation (Mazari et al., 2020).

7. Limitations, assumptions, and practical scope

The hypergraph PHPool formulation makes several explicit assumptions. It relies on a global hierarchical clustering built from shortest-path distances on the node-degree-preserving 2-section graph, uses hard clustering matrices $$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$3, and reconstructs higher-order structure through clique detection on coarsened pairwise adjacencies rather than through a separate learned edge-assignment matrix. The paper notes that $$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$4 dendrogram construction can be heavy for very large hypergraphs; suggested mitigations include approximate or distributed hierarchical clustering. It also notes that hard clustering is task-agnostic, and proposes as future directions the incorporation of node features and soft assignments learned end-to-end. Weighted hyperedges are handled naturally through the existing constructions, while directionality would require asymmetric generalizations (Wang et al., 8 Jun 2026).

The temporal predecessor has a different set of assumptions and limitations. It uses equal-length partitioning per level, average pooling within nodes, and a two-stage training pipeline in which frame features are extracted first and hierarchical pooling is learned afterward with SVM-based classification. The concatenation variant requires node alignment and is sensitive to misalignment at fine levels; the averaging kernel requires $$D_e = \mathrm{diag}(d(e_1), \dots, d(e_m)), \qquad d(e) = \sum_v H_{ve}.$$5 elementary kernel evaluations per video pair because of cross-breeding across node pairs; the EM-like multiple-kernel-learning solver can be computationally expensive due to alternating quadratic programs; and the approach is not fully end-to-end from pixels because it relies on precomputed 2D features. The paper identifies extension to longer videos, activity recognition, more scalable training, and potential end-to-end integration as future work (Mazari et al., 2020).

Taken together, these two formulations delimit the practical scope of PHPool. In the strict terminological sense of the acronym, PHPool is the hypergraph pooling operator built from a single dendrogram and paired with PHUnpool for U-Net-style encoder–decoder architectures. In a broader architectural sense, the label also captures a family of parallel hierarchical aggregation schemes in which multiscale branches are constructed together and merged by globally constrained operators rather than by sequential local decisions (Wang et al., 8 Jun 2026).