Rigorous justification of the replica-symmetric ERM characterization

Develop a mathematically rigorous proof of the replica-symmetric characterization of the global minimizers of the non-convex empirical risk for the single-hidden-unit autoencoder on the spiked cumulant model at fixed sample ratio α = n/d, including extending existing results for non-convex generalized linear models and proving that the replicon condition holds in this setting.

Background

The paper derives an asymptotic characterization of the empirical risk minimizer using the replica method under a replica-symmetric (RS) assumption. Because the ERM is non-convex, RS could be violated; if so, the stated formula would provide only a lower bound.

A rigorous derivation would require extending recent proofs of the replica formula for non-convex generalized linear models to the spiked cumulant setting and verifying the replicon stability condition, which the authors note as technically challenging.

References

Making \cref{res:erm} mathematically rigorous poses a considerable technical challenge that would require considerable extension of the results in to the present model and to show that the so-called replicon condition described therein holds. It is thus left for future work.

— A solvable high-dimensional model where nonlinear autoencoders learn structure invisible to PCA while test loss misaligns with generalization  (2602.10680 - Mendes et al., 11 Feb 2026) in Section 5 (Autoencoder: Empirical risk minimization), paragraph following Result 5.1