A solvable high-dimensional model where nonlinear autoencoders learn structure invisible to PCA while test loss misaligns with generalization

Abstract: Many real-world datasets contain hidden structure that cannot be detected by simple linear correlations between input features. For example, latent factors may influence the data in a coordinated way, even though their effect is invisible to covariance-based methods such as PCA. In practice, nonlinear neural networks often succeed in extracting such hidden structure in unsupervised and self-supervised learning. However, constructing a minimal high-dimensional model where this advantage can be rigorously analyzed has remained an open theoretical challenge. We introduce a tractable high-dimensional spiked model with two latent factors: one visible to covariance, and one statistically dependent yet uncorrelated, appearing only in higher-order moments. PCA and linear autoencoders fail to recover the latter, while a minimal nonlinear autoencoder provably extracts both. We analyze both the population risk, and empirical risk minimization. Our model also provides a tractable example where self-supervised test loss is poorly aligned with representation quality: nonlinear autoencoders recover latent structure that linear methods miss, even though their reconstruction loss is higher.

• The paper demonstrates that nonlinear autoencoders recover hidden latent factors in a high-dimensional spiked cumulant model, overcoming PCA’s inability to detect higher-order dependencies.
• It employs Hermite expansions and analyzes various activation functions to reveal how Bayesian optimal estimators and AMP algorithms achieve weak recovery above a critical sample threshold.
• Empirical results show that lower reconstruction loss does not equate to better generalization, urging the development of evaluation metrics that directly assess latent representation quality.

Solvable High-Dimensional Model: Nonlinear Autoencoders Learn Higher-Order Structure Invisible to PCA

Model Construction and Theoretical Framework

This paper introduces a rigorous high-dimensional data model—specifically a spiked cumulant model—featuring two latent factors. One factor is recoverable via covariance-based methods (such as PCA), while the second is statistically dependent yet uncorrelated, appearing exclusively in higher-order moments. Linear autoencoders and PCA are provably unable to detect the second latent structure unless the latent variables are linearly correlated, in which case information is present in the covariance. The correlation exponent $k^\star$ is defined as the minimal integer $k$ for which $\mathbb{E}[\lambda^k\nu]\ne0$; a finite exponent ($k^\star\ge 2$) characterizes situations where higher-order dependence enables recovery. The nonlinear autoencoder studied uses tied encoder/decoder weights and varying activation functions, and is trained either by minimizing population risk or empirical risk.

Information-Theoretic and Algorithmic Recovery Results

The Bayesian optimal recovery problem and Approximate Message Passing (AMP) algorithms are analyzed. The Bayes-optimal estimator achieves weak recovery of both latent spikes above a threshold sample ratio $\alpha_c$, and AMP also achieves the same performance—implying no statistical-computational gaps given this data structure. Crucially, weak recovery of the "hidden" spike $\mathbf{v^\star}$ is possible whenever the correlation exponent is finite, and the sample complexity for weak recovery matches the Baik-Ben Arous-Péché transition for the leading eigenvalue of the empirical covariance.

Nonlinear autoencoders, equipped with nontrivial activation functions and trained via gradient flow or empirical risk minimization (ERM), are shown to recover both latent factors at comparable sample complexity. The theoretical analysis employs Hermite expansions to study the population loss dynamics and reveals which activations (ReLU, ELU, Swish, Sigmoid, GELU) effectively recover higher-order structure, in contrast to linear and certain non-linearities such as $\tanh$ under specific dependence structures.

Figure 1: Cosine similarity of the Bayes-optimal and ERM estimators with $\mathbf{u^\star}$ and $\mathbf{v^\star}$ for autoencoders with varied activation; non-linearities achieve higher correlation with the "hidden" spike at higher sample ratios, which PCA/linear methods fail to recover.

Empirical Risk Minimization and Activation Function Analysis

Through replica calculations, ERM behavior is characterized in the high-dimensional limit. The loss landscape remains non-convex, but numerical investigations reveal that gradient descent closely tracks theoretical predictions, with non-linear autoencoders aligning with both spikes. For activations such as ReLU, ELU, Swish, Sigmoid, GELU, the empirical overlap with $\mathbf{v^\star}$ becomes significant as soon as the sample complexity crosses the critical threshold, whereas linear autoencoders never display such correlation.

Figure 2: Train and test loss of non-linear autoencoders minus the linear baseline, highlighting that linear models (PCA) minimize reconstruction loss, but fail to recover the hidden spike; right: classification error in downstream task, demonstrating superior performance of non-linearities.

The paper further analyzes how various activation functions fundamentally affect the ability to capture the structure in higher-order moments. For example, with correlation exponent $k^\star=3$, only activations whose Hermite decomposition aligns with the data's dependence structure (such as $\tanh$, Sigmoid for $k=3$) achieve recovery, while the others fail.

Misalignment Between Test Loss and Representation Quality

A strong, theoretically substantiated claim of this work is the explicit demonstration that "lower test loss does not equate to better generalization." In the spiked cumulant model, nonlinear autoencoders recover latent structure invisible to PCA, yet incur a higher test loss. This phenomenon is directly tied to the Eckart–Young theorem: linear autoencoders (equivalent to PCA) minimize the squared reconstruction error but miss out on capturing higher-order statistical dependence. The nonlinear methods, though suboptimal in reconstruction error, succeed in learning the full latent structure, which is evidenced by downstream task performance (e.g., linear classification based on recovered representations).

This result challenges the widespread practice of using validation loss as a proxy for representation quality and advocates for evaluation metrics that directly probe latent structure—such as probing tasks, linear regression on downstream targets, or fine-tuning based on small labeled sets.

Implications and Future Directions

The practical implication is a clear warning against relying on reconstruction loss as a training or validation criterion in self-supervised learning settings when higher-order correlations exist in the data that linear models cannot access. For theoretical machine learning, this paper establishes a tractable, minimal model that rigorously demonstrates the advantage of nonlinearity in representation learning, overcoming the limitations of classical spiked covariance models.

Frameworks developed here open avenues for analytically studying shallow (and potentially deep) architectures for extracting higher-order statistics, as well as probing the limits of self-supervised evaluation (e.g., in contrastive and masked learning objectives). Extensions to multi-neuron autoencoders, richer latent dependency regimes, and analysis of replica symmetry breaking in optimization landscapes remain compelling directions.

Figure 3: Cosine similarity and scaling with dimension $d$ for ERM estimators, demonstrating convergence towards theoretical predictions and highlighting finite-size effects.

Figure 4: Train and test loss for $\sigma=\tanh$ activation; illustrates scaling behavior and potential mismatch between empirical optimization and theoretical ERM predictions at low sample ratios.

Conclusion

This work rigorously formalizes a high-dimensional regime where non-linear autoencoders provably extract latent structure invisible to PCA, using both theoretical and empirical evidence. It also provides a concrete model for studying the misalignment between reconstruction test loss and representation quality, with implications for the design, training, and evaluation of self-supervised and unsupervised learning systems. This minimal yet rich model serves as a new benchmark for understanding the role of nonlinearity in unsupervised representation learning and challenges conventional wisdom in training and validation protocols.