Subcritical maximal-local-update behavior under correlated initialization

Establish whether correlated ResNets with scaling exponent \(\gamma>1\), equivalently \(\lambda_L=L^{-\gamma}\), exhibit the same subcritical ordinary-differential-equation behavior as networks with zero output weights in the independent-initialization setting.

Background

For γ>1\gamma>1, the residual scaling is smaller than the L−1L^{-1} scale, and the paper does not analyze the resulting training dynamics for correlated initialization.

The conjecture is based on analogy with the independent-initialization theory, where the subcritical maximal-local-update regime asymptotically agrees with zero output-weight initialization.

References

By analogy, we conjecture a similar subcritical ODE behavior in our setting.

— Correlated initialization of deep residual networks  (2609.03589 - Benning et al., 3 Sep 2026) in Section 3.1.2, “A conjectural phase diagram,” bullet “Maximal local updates (Subcritical ODE)”