Normalized Likelihood Criteria for Model Selection in the Stochastic Block Model

Abstract: Estimating the number of communities is a fundamental problem in network analysis under the stochastic block model (SBM). In this paper, we study penalized estimators for this task based on normalized likelihood criteria. We show that a penalized estimator derived from the Normalized Maximum Likelihood (NML) is strongly consistent with a logarithmic penalty term, although its computation is intractable. To overcome this limitation, we consider the Normalized Maximum Complete Likelihood (NMCL) and the Decomposed Normalized Maximum Likelihood (DNML). The DNML admits an explicit formulation with cubic computational complexity in the number of nodes. We prove that the NMCL- and DNML- based estimators are strongly consistent for sparse networks in which the average node degree diverges with the network size. Empirical results show that the DNML estimator performs competitively with existing methods, particularly in unbalanced networks.

• The paper introduces normalized likelihood criteria (NML, NMCL, DNML) that guarantee strong consistency in estimating the number of communities in SBM.
• The methodology leverages penalized estimators with penalties balancing model complexity and labeling uncertainty, achieving cubic time complexity.
• Empirical evaluation on synthetic and real networks demonstrates DNML's robustness in handling sparsity and unbalanced community structures.

Strongly Consistent Model Selection via Normalized Likelihood in the Stochastic Block Model

Introduction

Determining the number of communities in networks modeled by Stochastic Block Models (SBMs) is a pivotal yet challenging unsupervised learning problem. The complexity arises not just from the latent nature of group assignments but from the network's sparsity and scale, which push both statistical and computational limits. This paper introduces and rigorously analyzes penalized estimators based on normalized likelihood criteria—specifically the Normalized Maximum Likelihood (NML), Normalized Maximum Complete Likelihood (NMCL), and Decomposed Normalized Maximum Likelihood (DNML)—for model order selection in SBMs, providing strong guarantees under general sparsity regimes and efficient algorithms for practical large-scale use (2604.10205).

Theoretical Framework and Consistency Results

SBM Model and Problem Formulation

The stochastic block model (SBM) posits that within a graph $A$ of $n$ nodes, the probability of an edge between $i$ and $j$ depends solely on the latent assignments $Z_i, Z_j$ to $k$ communities. The problem addressed is: for a single observed adjacency matrix $A$, estimate $k$, the true number of communities.

Model selection uses penalized estimators of the form:

$$\hat{k}_n(A) = \arg\max_{1 \leq k \leq n} \{ l_n(A, k) - \text{pen}(k, n) \}$$

where $l_n$ is a data-dependent criterion inspired by normalized likelihoods.

Normalized Likelihood Criteria

1. NML Criterion: Ratio of the maximum likelihood for the observed network to the maximized likelihood summed over all networks of size $n$0 for a given $n$1. This is strongly consistent with a $n$2 penalty, but intractable computationally.
2. NMCL Criterion: Uses the maximized complete likelihood (given a single community assignment) over all possible networks and assignments. While the numerator is tractable, the denominator remains intractable.
3. DNML Criterion: Decomposes normalization into two terms: one for the network given assignments and one for the labelings, resulting in a closed-form criterion computable in cubic time in $n$3.

Both NMCL and DNML estimators depend on estimated labelings—typically provided by community detection heuristics—and require penalties containing both a $n$4 term (model complexity) and a $n$5 term (labeling uncertainty).

Strong Consistency: The authors rigorously prove that all three (NML, NMCL, DNML) are strongly consistent as $n$6, assuming either dense graphs ($n$7) or the sparse regime where $n$8 and $n$9, matching the sharpest known recovery thresholds.

Computational Aspects

While NML and NMCL are theoretically attractive, they require sums over exponentially many networks and/or labelings, making them infeasible. The key algorithmic contribution is the explicit DNML formula, which leverages reduction to products of multinomial normalizing constants, each computable in linear time in their counts, together yielding cubic time complexity for the full estimator.

The authors benchmark both the core DNML computation and entire pipeline (including community detection) using Fast-Greedy and Spectral Clustering (Figure 1).

Figure 1: Average runtime of the penalized estimator: DNML computation alone (red) and including community detection (green), for Fast-Greedy clustering.

The DNML estimator thus stands out in practical scalability for large $i$0, particularly when paired with efficient community detection algorithms.

Empirical Evaluation

Synthetic Networks

Across a suite of synthetic experiments, the DNML estimator is contrasted with established baselines: penalized maximum likelihood (PML), cross-validation (NCV/ECV), spectral methods (GFit, BHMC), corrected BIC (CBIC), and integrated likelihood (IL). Three main scenarios are investigated:

1. Scaling with $i$1: For fixed within ($i$2) and between ($i$3) community probabilities and $i$4 (Figure 2–3), DNML approaches $i$5 as $i$6 increases, with underestimation for small $i$7 that disappears at moderate sizes. Notably, DNML converges more rapidly than competitors in unbalanced settings.

Figure 2: DNML's average estimated $i$8 vs. $i$9 in balanced networks.

Figure 3: DNML's average estimated $j$0 vs. $j$1 in unbalanced networks ($j$2).

2. Detection under weak communities: At fixed $j$3, as $j$4, all methods degrade, but DNML displays superior robustness in unbalanced cases (Figure 4).

Figure 4: Estimated $j$5 as between-community probability $j$6 increases; right panel: unbalanced network.

3. Sparse Regime: DNML remains accurate as sparsity increases up to the threshold $j$7, where most competitors fail.

Penalty Term Sensitivity: The simulations reveal DNML tends to underestimate when the penalty is too aggressive (large $j$8 term), while omitting the $j$9 penalty induces overestimation. A log-only penalty performs better in moderate-$Z_i, Z_j$0 settings but lacks theoretical non-overestimation guarantees.

Real-World Networks

Across canonical benchmarks (Political Books, Dolphins, Karate, Football, Political Blogs), DNML achieves either exact or near-exact recovery of the known community counts, and crucially, outperforms other methods in the large-network, strong imbalance, and high-sparsity settings. Notably, in synthetic Football (1150), only DNML recovers the correct group count.

Implications and Future Directions

Theoretical Implications: This work establishes, for the first time, strong consistency for normalized likelihood criteria (specifically DNML) in SBM model order selection, for both dense and extremely sparse regimes. The theoretical analysis clarifies the tight connection between information complexity, community label uncertainty, and sparsity regime in dictating the shape of optimal penalty functions.

Practical Insights: The cubic-time DNML estimator is tractable for large graphs and empirically highly robust for unbalanced group sizes, a scenario where many existing methods fail. The penalty choice remains a delicate balance: log-only penalties risk overestimation while linear-in-$Z_i, Z_j$1 penalties may induce conservative underestimation.

Open Problems: While empirical results suggest log-only penalized DNML is effective in practical moderate-$Z_i, Z_j$2 cases, proving non-overestimation for minimal penalties remains open. Extending this methodology to degree-corrected, multilayer, and dynamic SBMs—using recent theoretical extensions—appears feasible and will further broaden impact.

Conclusion

This paper advances both theory and practice for selecting the number of communities in SBMs, presenting normalized likelihood criteria that are strongly consistent under the broadest known regime and formulating an explicit, scalable estimator (DNML) with demonstrably competitive or superior empirical performance. The proposed approach provides a statistically principled and computationally efficient solution for model selection in large, potentially sparse and unbalanced networks, paving the way for further developments in unsupervised network analysis (2604.10205).