Hierarchical Graph Information Bottleneck (HGIB)

• HGIB is a framework that applies the information bottleneck principle to hierarchical graph representations, compressing redundant features while preserving task-relevant information.
• It leverages multi-scale pooling and subgraph recognition techniques to balance compression and predictive power across layers, effectively handling noisy data.
• Variants such as HIBPool, GIB, and hypergraph applications demonstrate HGIB’s versatility, with reported improvements in recommendation HR/NDCG and prognosis accuracy.

Searching arXiv for the cited HGIB-related papers and close variants. Hierarchical Graph Information Bottleneck (HGIB) denotes graph-based formulations of the information bottleneck principle in which a representation is compressed across a hierarchy while preserving task-relevant information. In the recent literature, the term is used explicitly for a model-agnostic Hierarchical Graph Information Bottleneck framework for multi-behavior recommendation that learns compact yet sufficient representations and introduces a Graph Refinement Encoder for learnable edge pruning (Zhang et al., 21 Jul 2025). Related graph-learning formulations include Structure-Aware Hierarchical Graph Pooling using Information Bottleneck (HIBPool), which performs community-based hierarchical graph pooling (Roy et al., 2021), and Graph Information Bottleneck (GIB), which recognizes an IB-subgraph that is informative about the label while remaining compressive with respect to the original graph (Yu et al., 2020). A separate use of the acronym appears in Alzheimer’s disease prognosis, where HGIB denotes Hypergraph Information Bottleneck rather than Hierarchical Graph Information Bottleneck (Wang et al., 2023). A broader conceptual analogue treats layered corporate communication as a hierarchy of information bottlenecks with skip connections (Gordon, 2022).

1. Definition, scope, and shared objective

Across these works, the bottleneck principle is instantiated as a tradeoff between preserving information relevant to a target and discarding redundant or noisy information. The classic form is written as

$$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X}),$$

where minimizing $I(X;\hat{X})$ encourages compression and maximizing $I(Y;\hat{X})$ preserves predictive content (Gordon, 2022). Graph and hypergraph variants preserve this structure while changing the object being compressed: a pooled community representation, a subgraph, a hierarchical encoder state, or a hypergraph latent code.

Variant	Setting	Objective
Classic IB in a hierarchy (Gordon, 2022)	Layered communication system	$$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$
HIBPool (Roy et al., 2021)	Hierarchical graph pooling	$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$
GIB (Yu et al., 2020)	Subgraph recognition	$I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$
HGIB for recommendation (Zhang et al., 21 Jul 2025)	Hierarchical multi-behavior encoders	$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$
HGIB for prognosis (Wang et al., 2023)	Hypergraph neural learning	$[-I(Y;Z^l)+\beta I(X;Z^l)]$

The meaning of hierarchy differs across papers. In HIBPool, hierarchy is a stack of graph coarsening steps over communities. In GIB, the hierarchy is structural, from graph to subgraph to label, written as $G \rightarrow G_{\text{sub}} \rightarrow Y$. In the recommendation formulation, hierarchy is an encoder stack over unified, behavior-specific, and behavior-component representations. In the corporate analogue, hierarchy is the chain $L_1 \to L_2 \to \dots \to L_N$ with bounded attention at each layer. This suggests that HGIB is better understood as a family of information-theoretic design patterns than as a single canonical architecture.

2. Community-based hierarchical graph bottlenecks

HIBPool is the clearest graph-pooling realization of an HGIB-like idea: it learns a graph-level representation that is structurally aware, hierarchical, minimal but sufficient, and robust (Roy et al., 2021). The framework operates layer by layer on graphs,

$I(X;\hat{X})$0

and its pipeline is explicit: detect communities with Louvain modularity maximization, propagate features by

$I(X;\hat{X})$1

apply DiP-Readout, and construct the pooled graph via

$I(X;\hat{X})$2

The assignment matrix $I(X;\hat{X})$3 maps nodes to communities, so each pooled node corresponds to a discovered subgraph/community rather than to an arbitrary top-$I(X;\hat{X})$4 node selection.

The information bottleneck objective is imposed on the pooling summary $I(X;\hat{X})$5:

$I(X;\hat{X})$6

The paper makes a community-dependence assumption: each pooled representation depends only on its own community and the intra-community node features. The compression term is instantiated through a variational upper bound similar to GIB, with Gaussian parameters for node/community representations, while the predictive term is approximated by cross-entropy. The graph-level summary is then

$I(X;\hat{X})$7

A central technical component is Discriminative Pooling Readout (DiP-Readout). The method is motivated by the observation that simple Sum/Mean/Max/Min pooling can fail when node features become homogeneous. DiP-Readout therefore injects topological centrality measures such as degree, clustering coefficient, and betweenness through a centrality matrix $I(X;\hat{X})$8, normalizes centrality scores within each community, concatenates centrality-scaled embeddings,

$I(X;\hat{X})$9

and aggregates each community using sum, mean, max, and min before an MLP produces

$I(Y;\hat{X})$0

This yields a structure-aware community representation rather than a feature-only aggregation.

The hierarchical aspect appears both in repeated coarsening and in MHIBPool, which varies Louvain’s resolution via multi-scale modularity

$I(Y;\hat{X})$1

with $I(Y;\hat{X})$2 and reduction to standard modularity when $I(Y;\hat{X})$3. Empirically, HIBPool reports strong results on ENZYMES, DD, PROTEINS, NCI1, NCI109, and FRANKENSTEIN, including 83.45 ± 0.73 on NCI1 and 79.66 ± 0.72 on NCI109, while MHIBPool attains 75.0 ± 1.20 on ENZYMES and 83.48 ± 0.90 on NCI1. Under Gaussian feature perturbation, DIFFPOOL and ASAP degrade sharply, whereas HIBPool on NCI1 drops from 83.45 to 79.35 at perturbation ratio 2.0.

3. From graph compression to the IB-subgraph

GIB provides an important precursor by moving the bottleneck from latent vectors to explicit graph structure (Yu et al., 2020). The task is subgraph recognition: given a graph $I(Y;\hat{X})$4 and label/property $I(Y;\hat{X})$5, find a subgraph $I(Y;\hat{X})$6 that is as informative as possible about $I(Y;\hat{X})$7 while being compact and avoiding redundant or noisy structure. The defining objective is

$I(Y;\hat{X})$8

The paper also gives the constrained form

$I(Y;\hat{X})$9

with Lagrangian

$$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$0

The predictive term is rewritten so that maximizing $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$1 becomes equivalent to maximizing $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$2, approximated variationally by $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$3 and implemented through cross-entropy for classification or mean squared error for regression. The compression term is harder because mutual information over irregular graph objects is intractable. GIB addresses this with the Donsker–Varadhan representation,

$$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$4

and a sample-based estimator over positive pairs $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$5 and negative pairs $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$6, $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$7.

Optimization is formulated as a bi-level problem. The inner loop trains the statistics network $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$8 to maximize the mutual-information estimate. The outer loop updates the subgraph generator and predictor. Since node or edge selection is discrete, the paper relaxes subgraph selection to probabilities produced by a GNN + MLP, with each node receiving a 2-way assignment to $$\mathcal{L}[p(\hat{x}|x)] = I(X;\hat{X})-\beta I(Y;\hat{X})$$9 or its complement. A connectivity loss

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$0

stabilizes optimization and encourages compact, well-separated subgraphs rather than degenerate selections.

The empirical roles of the IB-subgraph are threefold. First, it improves graph classification when plugged into GCN, GraphSAGE, GIN, and GAT. Second, it serves interpretation: on QED, DRD2, HLM-CLint, and MLM-CLint, GIB yields the lowest absolute property bias and fewer disconnected pieces. Third, it performs graph denoising: on noisy MUTAG, it improves recall of true edges and precision of predicted real edges. In HGIB terms, this is a structural bottleneck that learns an informative and compressive graph component rather than only a pooled embedding.

4. HGIB as a model-agnostic framework for multi-behavior recommendation

The most direct use of the name Hierarchical Graph Information Bottleneck is the multi-behavior recommendation framework of 2025 (Zhang et al., 21 Jul 2025). The problem is formalized as a set of bipartite graphs

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$1

where one behavior $$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$2 is the target behavior and the rest are auxiliary behaviors. The framework is motivated by two stated difficulties: severe distribution disparities across behaviors and negative transfer effects caused by noise in auxiliary behaviors. Auxiliary behaviors such as views are denser than purchases and can create popularity bias, while sparse target behaviors can induce overfitting on sparse targets. At the same time, auxiliary interactions may be accidental or exploratory, so naïve aggregation can propagate noise.

HGIB is described as compatible with either the cascading paradigm or the parallel paradigm, but the concrete instantiation uses a parallel hierarchical backbone based on MULE. The instantiated hierarchy contains a unified encoder over all behaviors, behavior-specific encoders, behavior-component encoders for intersections and differences such as $$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$3 and $$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$4, and a fusion layer using target attention. The bottleneck objective over encoder layers is

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$5

Because direct mutual information optimization is intractable, the framework splits the objective into preservation and compression terms. The preservation side uses an InfoNCE-based lower bound,

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$6

yielding

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$7

and, in the concrete instantiation,

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$8

The compression side uses HSIC,

$$[- I(Y;S^{(l)}) + \beta I(A^{(l)},X^{(l)};S^{(l)})]$$9

with

$I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$0

and a more detailed instantiated form over the unified, behavior-specific, and behavior-component encoders.

To address noisy auxiliary interactions, HGIB introduces the Graph Refinement Encoder (GRE). For an edge $I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$1, GRE computes

$I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$2

and prunes by thresholding:

$I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$3

Because this decision is non-differentiable, the framework uses the Gumbel-softmax reparameterization trick, followed by LightGCN-style graph aggregation on the refined graph. The full loss is

$I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$4

with Adam, learning rate $I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$5, batch size 1024, max 100 epochs, default $I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$6, $I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$7, $I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$8, and $I(Y,G_{\text{sub}}) - \beta I(G,G_{\text{sub}})$9 on Taobao/Tmall and 0.5 on Jdata.

The reported evaluation covers Taobao, Tmall, and Jdata, with target behavior buy and full-ranking HR@10 and NDCG@10. HGIB achieves the best results on all three datasets. Compared with the strongest baseline, the reported numbers are Taobao: HR@10 0.2203 vs 0.1939, NDCG@10 0.1214 vs 0.1109; Tmall: HR@10 0.2427 vs 0.2109, NDCG@10 0.1287 vs 0.1165; Jdata: HR@10 0.6552 vs 0.6174, NDCG@10 0.4747 vs 0.4559. The relative improvements are +13.62% HR, +9.47% NDCG on Taobao, +15.08% HR, +10.47% NDCG on Tmall, and +6.12% HR, +4.12% NDCG on Jdata. The framework also improves the base hierarchical backbone as well as external backbones such as AutoDCS and MULE. In industrial experiments on WeChat News recommendation and Live stream recommendation, HGIB outperforms MULE by +14.85% HR, +17.69% NDCG and +5.51% HR, +7.30% NDCG, respectively. A 15-day online A/B test on WeChat news recommendation reports +0.32% CTR $$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$0 and +1.65% SSR $$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$1.

5. Hypergraph Information Bottleneck and the ambiguity of the acronym

A common source of confusion is terminological: HGIB in Alzheimer’s disease prognosis stands for Hypergraph Information Bottleneck, not Hierarchical Graph Information Bottleneck (Wang et al., 2023). The task is MCI conversion prognosis within a fixed two-year window using baseline multi-modal information. The framework represents patient cohorts as an undirected attributed hypergraph

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$2

where vertices are patients, hyperedges connect subsets of patients, and the incidence matrix satisfies

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$3

Hyperedges are constructed by a feature-space k-nearest-neighbor strategy for each modality, yielding modality-specific hyperedge groups

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$4

which are concatenated as

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$5

The model uses spatial hypergraph convolution. Information moves from vertices to hyperedges and back,

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$6

which captures higher-order dependencies among patients. The information bottleneck objective is then imposed layerwise:

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$7

The predictive term is approximated by cross-entropy, while the compression term assumes Bernoulli-distributed binary latent variables and uses

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$8

The total loss combines cross-entropy, focal loss, and the HGIB regularizer:

$$\sum_{l=1}^L [I(\boldsymbol{E}^{l};\boldsymbol{Y}) - \beta I(\boldsymbol{E}^{l}; \boldsymbol{E}^{l-1})]$$9

with $[-I(Y;Z^l)+\beta I(X;Z^l)]$0, $[-I(Y;Z^l)+\beta I(X;Z^l)]$1, $[-I(Y;Z^l)+\beta I(X;Z^l)]$2, and $[-I(Y;Z^l)+\beta I(X;Z^l)]$3. Training uses PyTorch with the DHG library, one NVIDIA A100 GPU, Adam, 2000 epochs, learning rate $[-I(Y;Z^l)+\beta I(X;Z^l)]$4, and a hypergraph neighborhood size of 20 neighbors.

On ADNI, after filtering for complete multi-modal samples, the study uses 248 patients from ADNI-2 and reports AUC, PPV, and NPV averaged over five independent runs. HGIB achieves NC AUC $[-I(Y;Z^l)+\beta I(X;Z^l)]$5, MCI AUC $[-I(Y;Z^l)+\beta I(X;Z^l)]$6, AD AUC $[-I(Y;Z^l)+\beta I(X;Z^l)]$7, Average AUC 0.7595, Average PPV 60.98, and Average NPV 76.19. Baseline average AUC values are 0.6275 for DHGNN, 0.7346 for HGNN, and 0.7437 for HGNN+. Under label-efficiency settings of 80%, 60%, and 40% labeled data, HGIB remains best overall or best on key metrics. Under robustness tests, including randomly dropping 20% of hyperedges and injecting Gaussian feature noise with $[-I(Y;Z^l)+\beta I(X;Z^l)]$8, HGIB remains above HGNN+ in both attacked settings. The paper also reports a Friedman test $[-I(Y;Z^l)+\beta I(X;Z^l)]$9, Kendall’s coefficient of concordance $G \rightarrow G_{\text{sub}} \rightarrow Y$0, and pairwise Wilcoxon tests.

6. Conceptual bridges, misconceptions, and limitations

The broadest conceptual bridge comes from the application of the bottleneck principle to layered organizations (Gordon, 2022). There, a firm is treated as a hierarchical communication system in which raw information $G \rightarrow G_{\text{sub}} \rightarrow Y$1 is transformed into compressed messages $G \rightarrow G_{\text{sub}} \rightarrow Y$2 or $G \rightarrow G_{\text{sub}} \rightarrow Y$3 on the way to a decision-relevant target $G \rightarrow G_{\text{sub}} \rightarrow Y$4. The hierarchy is modeled as

$G \rightarrow G_{\text{sub}} \rightarrow Y$5

with a Markov-chain interpretation and the data processing inequality

$G \rightarrow G_{\text{sub}} \rightarrow Y$6

The paper then extends the strict chain by discussing side channels and skip connections, analogous to ResNets, DenseNets, and other deep networks where shortcut paths help preserve signal and prevent vanishing gradients. In graph terms, this means a hierarchy need not be a pure tree or chain; it may include main reporting edges, lateral or upward shortcut edges, and side channels that preserve raw or less-compressed information.

Several misconceptions follow from this literature. First, HGIB is not a single algorithm shared across all graph domains. The recommendation model is explicitly model-agnostic and plugs into hierarchical recommenders such as AutoDCS and MULE, whereas HIBPool is a community-based pooling operator, GIB is a subgraph recognizer, and the Alzheimer’s model uses the same acronym for a hypergraph prognosis framework. Second, bottlenecking on graphs is not restricted to graph coarsening. It may act on pooled communities, explicit subgraphs, hierarchical encoder states, or layerwise hypergraph latents. Third, robustness is not attributed to a single mechanism. HIBPool ties it to IB-guided community pooling and structure-aware readout, whereas the recommendation HGIB combines preservation/compression losses with the explicit denoising of GRE.

The limitations are correspondingly heterogeneous. The organizational paper is mainly conceptual and illustrative rather than a formal graph-learning model: it provides no empirical estimation of mutual information in real organizations, no full probabilistic graphical model of the hierarchy, and does not develop a generalized HGIB algorithm or architecture (Gordon, 2022). GIB emphasizes that mutual information on irregular graph data is hard to optimize and requires a mutual-information estimator, a bi-level optimization scheme, and a connectivity loss to stabilize training (Yu et al., 2020). HIBPool assumes that each pooled representation depends only on its own community and intra-community node features, which makes community detection central to the method’s behavior (Roy et al., 2021). The recommendation HGIB is tied to hierarchical multi-behavior settings, and the paper explicitly notes that if a method has no hierarchical structure, HGIB can degenerate into a standard information bottleneck formulation (Zhang et al., 21 Jul 2025). The prognosis HGIB is scoped to multi-modal Alzheimer’s conversion prediction within a two-year window and therefore should not be conflated with the graph-recommendation use of the acronym (Wang et al., 2023).

Taken together, these works define HGIB less as a single architecture than as an information-theoretic program for structured predictive compression: compress lower-level, noisy, or redundant graph information; preserve what is sufficient for the downstream target; and, when strict hierarchies are lossy, introduce multi-path communication or explicit refinement mechanisms to mitigate information loss.