Order-four critical growth law

Determine the critical-depth growth law for the fourth input-derivative variance of the wide scalar-input fully connected network, including whether it follows the polynomial growth pattern established for derivative orders one through three.

Background

The paper proves depth-invariant first-derivative variance, linear second-derivative variance growth, and quadratic third-derivative variance growth at criticality. It also displays fourth-order simulations only to check boundedness under residual scaling.

The critical behavior of the fourth-order variance in the ordinary fully connected network is not derived. The conclusion explicitly identifies the absence of an order-four growth law as one of the unresolved questions left by the work.

References

Two questions are left open. Conjecture 1 asserts that the closure persists at every order, with coefficients polynomial in χ1, . . . , χk and no other moment of the activation. The evidence is orders one through three, where each cancellation consumes one of the fixed-point identities, and the general case is unproved. The order-four variance is reported only as a boundedness check; no order-four growth law is claimed here.

— Critical initialization destabilizes higher input derivatives in wide scalar-input networks  (2609.09244 - Singh et al., 8 Sep 2026) in Section 9, Conclusions