Likelihood Ratio Scoring via Block Rejection

• The paper introduces ε-regularization to stabilize the likelihood ratio statistic in sparse stochastic block models, ensuring valid hypothesis testing.
• It applies a Monte Carlo approximation to efficiently manage the sum over 2^n labelings, yielding asymptotic power-Poisson distributions with rigorous error control.
• The method effectively distinguishes Erdős–Rényi graphs from balanced community structures, outperforming traditional spectral and subgraph-count tests in high-SNR regimes.

Likelihood ratio scoring with block rejection refers to a rigorous statistical methodology for hypothesis testing in network data, specifically for distinguishing between an Erdős–Rényi (ER) random graph and a balanced two-community stochastic block model (SBM) in the bounded degree regime. The standard likelihood ratio (LR) approach degenerates in sparse settings due to the asymptotic orthogonality of the probability measures when the signal-to-noise ratio exceeds a certain threshold. To address this, an $\varepsilon$–regularization is introduced to stabilize the LR statistic, allowing for valid inference through block rejection rules. The resulting test yields asymptotic distributions characterized by power-Poisson laws, and achieves robust performance via Monte Carlo approximation, with strong theoretical and empirical guarantees in the high-SNR regime (Yuan et al., 2018).

1. Problem Formulation and Model Definitions

Consider the testing problem where an observed undirected graph $G$ on $n$ vertices is to be distinguished between:

• $H_0$: $G \sim G(n, p_0)$ (Erdős–Rényi, with $p_0=(a+b)/(2n)$)
• $H_1$: $G \sim G(n, a, b)$ (balanced bisection SBM), where each vertex is labeled $o_u \in \{\pm 1\}$ independently, and

$$\Pr(A_{uv}=1 \mid o) = \begin{cases} a/n, & o_u = o_v \ b/n, & o_u \neq o_v \end{cases} \quad a > b > 0$$

The “signal‐to‐noise ratio” (SNR) is defined as $\kappa = \frac{(a-b)^2}{2(a+b)}$. When $\kappa \ge 1$ and $a, b$ are fixed (bounded degree regime), the classical LR statistic fails due to the lack of contiguity between the distributions. The problem forms the foundation for hypothesis testing in community detection and the determination of the number of communities.

2. Regularized Likelihood Ratio Statistic

An $\varepsilon$–regularization is introduced, modifying the intra-group and inter-group edge probabilities:

$$a_\varepsilon = a - \varepsilon, \quad b_\varepsilon = b + \varepsilon, \quad 0 < \varepsilon < \frac{a-b}{2}, \;\; \frac{(a-b_\varepsilon)^2}{2(a+b)} < 1$$

The regularized likelihood under the SBM model for a label assignment $o$ is

$$\ell_1^\varepsilon(A \mid o) = \sum_{u < v}\Big\{A_{uv} \log P^\varepsilon_{uv}(o) + (1 - A_{uv}) \log(1 - P^\varepsilon_{uv}(o))\Big\}$$

where $P^\varepsilon_{uv}(o) = a_\varepsilon/n$ if $o_u = o_v$, $b_\varepsilon/n$ if $o_u \neq o_v$. The ER log-likelihood is $\ell_0(A)$, and the $\varepsilon$–regularized LR statistic is the average over all labelings:

$$Y_n^\varepsilon = \frac{1}{2^n} \sum_{o \in \{\pm1\}^n} \exp\left[\ell_1^\varepsilon(A \mid o) - \ell_0(A)\right]$$

Regularization ensures the ratio $(a_\varepsilon-b_\varepsilon)/(a+b)$ is strictly less than one, suppressing the explosive variance observed with the standard LR in the high-SNR bounded-degree setting.

3. Asymptotic Power-Poisson Laws

As $n \to \infty$ and $\kappa > 1$, the asymptotic distributions of $Y_n^\varepsilon$ are infinite “power-Poisson” products. For each cycle length $m \ge 3$:

$$\lambda_m^\varepsilon = \frac{1}{2m}\left(\frac{a_\varepsilon + b_\varepsilon}{2}\right)^m, \quad \delta_m^\varepsilon = \left(\frac{a_\varepsilon-b_\varepsilon}{a+b}\right)^m$$

• Under $H_0$:

$$Y_n^\varepsilon \xrightarrow{d} W_\varepsilon = \prod_{m=3}^\infty (1+\delta_m^\varepsilon)^{Z_m} \exp(-\lambda_m^\varepsilon \delta_m^\varepsilon), \quad Z_m \, \stackrel{\text{i.i.d.}}{\sim} \, \operatorname{Poisson}(\lambda_m^\varepsilon)$$

• Under $H_1$ (with block-signal parameter $\delta_m = (\frac{a-b}{a+b})^m$):

$$Y_n^\varepsilon \xrightarrow{d} W_1 = \prod_{m=3}^\infty (1+\delta_m^\varepsilon)^{\widetilde Z_m} \exp(-\lambda_m^\varepsilon \delta_m^\varepsilon), \quad \widetilde Z_m \, \stackrel{\text{i.i.d.}}{\sim} \, \operatorname{Poisson}(\lambda_m^\varepsilon(1+\delta_m))$$

These results rely on a Janson-type contiguity criterion and sufficient control over mixed moments of cycle counts and $Y_n^\varepsilon$.

4. Rejection Rule and Error Rates

Given a desired significance level $0 < \alpha < 1$, compute the $(1-\alpha)$-quantile $w_\alpha$ of $W_\varepsilon$ under $H_0$ ($\mathbb{P}(W_\varepsilon \le w_\alpha) = 1-\alpha$). The block rejection rule:

• Reject $H_0$ if $Y_n^\varepsilon > w_\alpha$ achieves

$$\lim_{n \to \infty} \mathbb{P}_{H_0}(Y_n^\varepsilon > w_\alpha) = \alpha$$

$$\lim_{n \to \infty} \mathbb{P}_{H_1}(Y_n^\varepsilon \le w_\alpha) = \mathbb{P}(W_1 \le w_\alpha)$$

Power analysis reveals that for suitable regularization parameters and growing average degree, the test is asymptotically powerful whenever $\kappa \to \infty$.

5. Monte Carlo Approximation and Computational Considerations

Direct evaluation of $Y_n^\varepsilon$ is infeasible since it requires summing over $2^n$ labelings. A Monte Carlo (MC) estimator using $M$ i.i.d. labelings achieves

$$\widehat{Y}_n^\varepsilon = \frac{1}{M} \sum_{i=1}^M \prod_{u < v} \left(\frac{P_{uv}^\varepsilon(o^{(i)})}{p_0}\right)^{A_{uv}} \left(\frac{1-P_{uv}^\varepsilon(o^{(i)})}{1-p_0}\right)^{1 - A_{uv}}$$

with computational cost $O(M n^2)$, where $M \gg \exp(2 \kappa_\varepsilon n)$ and $$\kappa_\varepsilon = (a_\varepsilon - b_\varepsilon)^2 / (2(a+b))$$ is required for negligible MC error. This allows practical application for moderate system sizes, underlining the method's potential in sparse network regimes.

6. Empirical Studies and Performance Benchmarking

Simulations for values $a = 2.5 + c$, $b = 2.5 - c$, and $c \in \{2.10, 2.15, 2.25, 2.35\}$ correspond to SNR $\kappa \approx 1.76, 1.85, 2.03, 2.21$ with $n \in \{20,30,40,45\}$. With appropriately chosen $\varepsilon$, the empirical size at level $\alpha=0.05$ matches the theoretical prediction, and power increases with $n$ and $\kappa$. The regularized LR procedure outperforms the spectral test of Bickel–Sarkar and the subgraph-count test of Gao–Lafferty in these sparse regimes.

On real data, e.g., the “political books” co-purchase network ($n=105$), the test shows high success rates (83%–100%) in correctly rejecting the null when distinct communities are merged, again exceeding the performance of competing methods in sparse graphs.

7. Extensions, Limitations, and Future Directions

The $\varepsilon$–regularization is critical for restoring statistical contiguity and mitigating the erratic behavior of the standard LR statistic in SBMs with bounded average degree. No test is possible for $\kappa < 1$ due to information-theoretic lower bounds. The MC approach's computational cost remains exponential in the worst case; further analysis of deterministic or mean-field approximations is needed. Extensions to multi-community SBMs, degree-corrected models, and connections with semidefinite and spectral relaxations are posited as promising avenues for future work. The likelihood ratio scoring with block rejection framework establishes a principled, theoretically robust paradigm for community hypothesis testing in sparse networks (Yuan et al., 2018).