Structural Entropy Minimization Algorithm

• Structural Entropy Minimization Algorithm is a graph-based optimization method that reduces the expected code length of random walk transitions to reveal hidden clusters.
• It reveals hierarchical structures and community patterns in complex networks, aiding applications like image segmentation, traffic forecasting, and event detection.
• Greedy and iterative merge heuristics approximate the NP-hard minimization problem, ensuring scalable, privacy-preserving clustering and compression in multi-relational settings.

Structural Entropy Minimization Algorithm

The structural entropy minimization algorithm encompasses a family of graph-based discrete optimization methods whose objective is to discover clusterings, partitions, or hierarchical structures by minimizing the information-theoretic "structural entropy" associated with random walks on graphs. At its core, structural entropy quantifies the expected code length required to describe a random walker's transitions under a given partition or encoding tree, and its minimization reveals organization, redundancy, or modularity within complex networks, including graphs with multi-relational or heterogeneous edges. Minimizing this entropy is widely applied in network compression, community detection, unsupervised clustering, event detection, image segmentation, structural inference, and more, under classical, privacy-preserving, and multi-relational settings.

1. Formal Definitions of Structural Entropy

Structural entropy in its most classical form for an undirected (possibly weighted) graph $G = (V, E, w)$, with degree $d(v)$ and graph volume $\mathrm{vol}(V)$, is defined through the information content of stationary random walks and code trees ("encoding trees") over the node set.

• One-dimensional (1D) structural entropy:

$$H^{(1)}(G) = -\sum_{v\in V} \frac{d(v)}{\mathrm{vol}(V)} \log_2 \frac{d(v)}{\mathrm{vol}(V)}$$

This is the Shannon entropy of the endpoint distribution of a stationary random walk.

• Two-dimensional (2D) structural entropy:

For a partition $P = \{C_1, ..., C_k\}$,

$$H^{(2)}(G; P) = -\sum_{i=1}^k \frac{g(C_i)}{\mathrm{vol}(V)} \log_2 \frac{\mathrm{vol}(C_i)}{\mathrm{vol}(V)} + \sum_{i=1}^k \frac{\mathrm{vol}(C_i)}{\mathrm{vol}(V)}H^{(1)}(G_{C_i})$$

where $g(C_i)$ is the total cut-weight from $C_i$ to $V\setminus C_i$, and $H^{(1)}(G_{C_i})$ is the 1D entropy on the subgraph induced by $d(v)$0.

• Hierarchical and Multi-relational Structural Entropy:

The formalism extends to hierarchical (multi-level encoding trees) and multi-relational graphs, where transitions also account for edge types and stationary distributions over relation sets (Cao et al., 2024). In the multi-relational case, the entropy accounts for both node- and relation-transition probabilities.

This information-theoretic formalism brings several key properties: (i) entropy decreases under coarsening if and only if merged modules provide compression, (ii) clustering is nonparametric—the number of clusters emerges from the data, and (iii) two-level (or higher-level) entropies guide not only partitions but also hierarchy discovery (Zou et al., 2023, Cao et al., 2023, Yang et al., 2024, Zhang et al., 31 Jul 2025).

2. Minimization Objectives and Theoretical Properties

The algorithmic goal is to find a partition or encoding tree $d(v)$1 that minimizes structural entropy:

$d(v)$2

for hierarchies, or

$d(v)$3

for partitions (Zou et al., 2023, Cao et al., 2023, Yang et al., 2024, Zhang et al., 31 Jul 2025).

This minimization is NP-hard in general for nontrivial trees/partitions, necessitating effective approximate algorithms. Key theoretical results include:

• Every merge in a coding tree strictly decreases $d(v)$4 or leaves it unchanged; thus, greedy minimization is monotonic and converges in finite steps (Yang et al., 2024).
• The minimizer always satisfies $d(v)$5, with equality only at the singleton partition.
• In the context of Stochastic Block Models, the partitioned structural entropy has a closed-form and a matching algorithmic compression scheme achieving the source-coding bound (Han et al., 2021).
• Multi-relational SE ("MrSE") captures additional structure by jointly minimizing over both node and relation transitions, outperforming single-relation SE in clustering and event detection tasks (Cao et al., 2024).

3. Greedy and Hierarchical Minimization Procedures

Due to the nonconvex and combinatorial nature of the problem, practical algorithms use greedy or iterative merge heuristics:

• Initialization: Each node forms its own module/cluster.
• Merging Stage: Iteratively compute the entropy reduction $d(v)$6 for all admissible merges (pairs of clusters or siblings in the encoding tree). Merge the pair with the largest $d(v)$7; repeat until no positive gain remains (Zou et al., 2023, Peng et al., 7 Aug 2025, Zeng et al., 2023).
• Parallelization and Hierarchical Strategies: To scale, clusters are grouped into subgraphs (of user-defined size $d(v)$8 or $d(v)$9), on which merges are performed in parallel. When no local merges yield improvement, $\mathrm{vol}(V)$0 or $\mathrm{vol}(V)$1 is doubled and the procedure repeats (Cao et al., 2023, Yang et al., 2024).

Pseudocode skeleton (greedy merge):

$\mathrm{vol}(V)$7

Complexity:

• Single-threaded O(N³) worst-case (all pairs considered), reduced to O(N² log N) with local restrictions or heap-based caching (Zou et al., 2023).
• Parallel block-wise updates achieve O(Nq) per outer iteration, and parallelism is "embarrassingly" available.

The process is illustrated in algorithmic frameworks for event detection (HISEvent, ADP-SEMEvent), dialogue partitioning (DSEM), traffic forecasting (MultiSPANS), image segmentation (SLED), and document summarization (MRGSEM-Sum) (Cao et al., 2023, Yang et al., 2024, Peng et al., 7 Aug 2025, Zou et al., 2023, Zeng et al., 2023, Zhang et al., 31 Jul 2025).

4. Extensions: Multi-relational, Privacy-preserving, and Specialized SE Minimization

Recent work extends the classical algorithm in several ways:

• Multi-relational SE Minimization: Transition tensors model the random walker choosing both relation and node types; the minimization proceeds analogously but requires stationary distributions over relations. This leads to MrSE, whose greedy merge procedures tightly generalize their single-relational analog (Cao et al., 2024, Zhang et al., 31 Jul 2025).
• Adaptive Differential Privacy: SE minimization is performed on privatized message graphs, with privacy preservation via adaptive mechanisms. Subgraph decomposition and parallel blockwise merging ensure efficient and privacy-preserving entropy minimization (Yang et al., 2024).
• Entropy-regularized Structural Inference: In deep structural prediction (e.g., differentiable matching via Sinkhorn), instability in annealing motivates adaptive, stability-controlled entropy minimization dynamics to avoid premature mode collapse, via drift-based runtime control (PH-ASC/EPH-ASC) (Liu, 30 Jan 2026).

5. Applications Across Domains

The structural entropy minimization algorithm underpins applications including:

Domain	SE Minimization Objective	Representative Papers
Community detection	Minimize 2D entropy for clusters	(Zou et al., 2023, Cao et al., 2024, Han et al., 2021)
Event detection	Unsupervised message graph clustering for events	(Cao et al., 2023, Yang et al., 2024)
Document summarization	Adaptive sentence clustering	(Zhang et al., 31 Jul 2025)
Dialogue structure	Partition into sub-dialogues	(Peng et al., 7 Aug 2025)
Image segmentation	Superpixel graph segmentation	(Zeng et al., 2023)
Optimal transport (structural inference)	Stability-controlled, annealed entropy minimization	(Liu, 30 Jan 2026)

Significant results include:

• Improved node clustering and event detection in multi-relational social and bibliographic networks by MrSE (Cao et al., 2024, Zhang et al., 31 Jul 2025).
• Efficient, privacy-preserving event detection with adaptive SE minimization on noisy social graphs (Yang et al., 2024).
• Enhanced interpretability and compression-optimality in Stochastic Block Model compression algorithms (Han et al., 2021).
• Robust traffic forecasting via spatial attention masking derived from SE-based road-network hierarchies (Zou et al., 2023).
• Improved image segmentation fidelity via multi-scale, entropy-minimizing superpixel graph partitioning (Zeng et al., 2023).
• Stability-controlled inference in entropy-regularized deep matching (Liu, 30 Jan 2026).

6. Hyperparameters, Limitations, and Practical Considerations

Principal hyperparameters include:

• Subgraph/block size ($\mathrm{vol}(V)$2 or $\mathrm{vol}(V)$3): Trade-off between merge granularity and runtime. Larger $\mathrm{vol}(V)$4 yields coarser merges but increased cost per round (Cao et al., 2023, Yang et al., 2024).
• Merge threshold (e.g., convergence $\mathrm{vol}(V)$5, parallel-merge ratio $\mathrm{vol}(V)$6): Affects result granularity and algorithmic convergence (Peng et al., 7 Aug 2025).
• Graph construction and edge-weighting: Preprocessing can affect entropy landscape and the quality of partitions (Zeng et al., 2023).
• For multi-relational methods, relation importance emerges from the stationary distribution; choice of relation integration (e.g., max, sum) influences results (Cao et al., 2024).

Limitations:

• SE minimization is NP-hard; standard algorithms find local minima, not global optima.
• In some settings, e.g., multi-relational summarization, relation-type integration is fixed (e.g., max-over-relations), leaving finer modeling as future work (Zhang et al., 31 Jul 2025).
• Scalability to giant graphs may require preprocessing, aggressive blockwise coarsening, or approximate nearest neighbor search.
• In the SBM context, the method assumes a known block partition; learning block assignments is not directly addressed by the compression algorithm (Han et al., 2021).

7. Connections to Information Theory and Broader Impact

SE minimization is fundamentally an information-theoretic approach: minimum entropy partitioning aligns with the minimum expected code length of random walks, and thus the lossy or lossless compression of structural patterns (Han et al., 2021). It generalizes source coding, clustering, and hierarchical modeling across networked data and is adaptive to various constraints, including privacy and multi-relational structure. Applications continue to broaden into spatiotemporal modeling, interpretable machine learning components, and privacy-aware data mining (Cao et al., 2023, Yang et al., 2024, Zhang et al., 31 Jul 2025).

The theoretical and empirical findings, as synthesized above, establish structural entropy minimization as a foundational method for unsupervised, interpretable, and scalable discovery of organization in graphs and relational data (Zou et al., 2023, Cao et al., 2024, Han et al., 2021, Liu, 30 Jan 2026, Zeng et al., 2023, Zhang et al., 31 Jul 2025, Peng et al., 7 Aug 2025).