Dimensional Collapse in Complex Systems

• Dimensional collapse is the reduction of effective state space where high-dimensional trajectories concentrate on a lower-dimensional manifold or discrete set of states.
• It manifests across disciplines—from photonic Ising machines using polarization dynamics to self-supervised learning where feature representations become rank-deficient.
• Quantitative indicators like covariance spectrum decay and effective rank help diagnose collapse, while orthogonality regularization and other remedies mitigate its negative effects.

Dimensional collapse denotes a phenomenon wherein the effective number of degrees of freedom, or the dimensionality of a system’s relevant state space, reduces drastically either dynamically or structurally. This occurs when high-dimensional trajectories or solutions, instead of exploring their whole ambient space, ultimately concentrate onto a much lower-dimensional subset, subspace, or fixed set of discrete states. The mechanisms and implications of dimensional collapse manifest in diverse domains: nonlinear photonic systems, self- and contrastive-supervised learning, federated optimization, Riemannian geometry, and gravitational physics. Despite the disciplinary breadth, dimensional collapse universally signals the loss—or useful condensation—of high-dimensional structure, with far-reaching consequences for representation, optimization, and physical criticality.

1. Mathematical Definition and Core Mechanisms

Dimensional collapse occurs when a system, initially able to explore a high-dimensional state space, self-organizes (either dynamically or through training) so that its asymptotic states reside on a lower-dimensional manifold or a small discrete set:

• Dynamical Example: In polarization-based Ising machines, each oscillator’s state is a vector $(S_1,S_2,S_3)/S_0$ on the Poincaré sphere. Through feedback and nonlinearity, the trajectory collapses onto one of two antipodal points, representing Ising spins, thereby reducing three degrees of freedom to a single binary variable (Chiavazzo et al., 11 Apr 2025).
• Statistical Learning Example: For a set of learned representations $Z \in \mathbb{R}^{N\times d}$, collapse implies the empirical covariance matrix $\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$ is (nearly) rank-deficient: most eigenvalues $\lambda_i$ are close to zero, so the representations lie in a low-dimensional subspace (Jing et al., 2021, Chen et al., 2023, Shi et al., 2022, He et al., 2024).

Collapse can be self-induced (e.g., through dynamics or loss minimization; no external control parameter is required as in the PIM (Chiavazzo et al., 11 Apr 2025)), or it may occur as a side effect or pathology (e.g., excessive regularization, data heterogeneity, excessive augmentation, or ill-posed optimization in machine learning).

2. Dimensional Collapse in Photonic Ising Machines

The "dimensional collapse" mechanism underpins the operation of the Polarization Ising Machine (PIM), which leverages the three-dimensional polarization degrees of freedom of optical pulses:

• State Space: Each pulse is described by its Stokes vector, a point on the unit Poincaré sphere, parameterizing normalized polarization $(S_1/S_0, S_2/S_0, S_3/S_0)$.
• Nonlinear Dynamics: The pulse evolves according to a nonlinear polarization oscillator (NPO) equation, with feedback coupling via Ising-like interactions implemented opto-electronically.
• Collapse Process: Under feedback, all trajectories are funneled to the S₁ axis, specifically $S_1/S_0 = \pm1$ with $S_2 = S_3 = 0$. This is the collapse from a 3D to a 1D submanifold—for the coupled case, precisely onto the discrete Ising configurations.
• Functional Role: The transient exploitation of $S_2$ and $S_3$ components enables the system to avoid local minima, thereby improving the likelihood of reaching the ground state of the Ising Hamiltonian compared to phase-only (1D) approaches (Chiavazzo et al., 11 Apr 2025).

This photonic dimensional collapse is robust, does not require adiabatic scheduling or external annealing, and is analytically characterized by the feedback dynamics yielding only two attractors in the S₁ direction.

3. Dimensional Collapse in Representation Learning

In contrastive and self-supervised representation learning, dimensional collapse describes the tendency of learned feature representations or model weights to concentrate on a subspace of reduced effective rank:

• Spectral Diagnostics: Compute the spectrum of the empirical feature covariance matrix; collapse is observed as only a few non-negligible eigenvalues (effective rank $r \ll d$).
• Origins: Causes include InfoNCE-like objectives ("uniformity–alignment trade-off"), excessive data augmentation (strong-augmentation collapse), and implicit low-rank bias from optimization dynamics (Jing et al., 2021, Chen et al., 2023, He et al., 2024).
• Consequences: Collapsed representations lose discriminative capacity and adversely affect downstream task accuracy.
• Mitigation Strategies: Orthogonality regularization (enforcing $Z \in \mathbb{R}^{N\times d}$0 on weights) resists collapse not only in final representations but throughout deeper layers, as empirically validated across CNN and transformer architectures (He et al., 2024). Alternative approaches directly maximize the representation coding rate (log-determinant regularization) or penalize inter-feature correlation (as in "FedDecorr" for federated learning) (Chen et al., 2023, Shi et al., 2022).

In collaborative filtering, dimensional collapse manifests as embeddings lying in a low-rank manifold, suppressing informative axes for user/item discrimination (Chen et al., 2023). In federated settings, collapse is driven by data heterogeneity and is inherited from local models to the global aggregate (Shi et al., 2022).

4. Dimensional Collapse in Curvature-Bounded Geometry

In Riemannian geometry, "collapse" refers to a sequence of $Z \in \mathbb{R}^{N\times d}$1-manifolds $Z \in \mathbb{R}^{N\times d}$2 (with uniform lower sectional curvature and bounded diameter) converging in Gromov–Hausdorff topology to a metric space $Z \in \mathbb{R}^{N\times d}$3 of dimension $Z \in \mathbb{R}^{N\times d}$4 (Yamaguchi, 2012, Roos, 2017):

• Collapse Classification: The topology of $Z \in \mathbb{R}^{N\times d}$5 is strongly constrained by the structure of the limit $Z \in \mathbb{R}^{N\times d}$6: for example, in 4D, the manifold may fiber over $Z \in \mathbb{R}^{N\times d}$7 with standard fiber types (spheres, tori, infranilmanifolds) determined by the codimension.
• Codimension One Characterization: In codimension-one collapse, the decrease in dimension is tightly linked to quantitative relationships between volume and injectivity radius at small scales, with the volume of geodesic balls decaying at the same rate as the injectivity radius (Roos, 2017).
• Fibration Theorems: Local and global bundle/fibration structures emerge generically, with singular fibers over the singular strata of $Z \in \mathbb{R}^{N\times d}$8 (Yamaguchi, 2012). This geometric collapse is essential to the metric classification of Riemannian and Alexandrov spaces.

5. Dimensional Collapse in Gravitational and Critical Phenomena

In gravitational physics, the term "dimensional collapse" has distinct but related significance:

• Gravitational Collapse: The fate of gravitationally collapsing matter—black hole vs. naked singularity—depends sensitively on the space’s dimension and physical parameters (Sarwe et al., 2012, Shaikh et al., 2018). Mathematical criteria for collapse end-states are dimension-independent at the level of local initial data.
• Critical Collapse in Higher-Curvature Gravity: Introduction of a new length scale (e.g., Gauss–Bonnet $Z \in \mathbb{R}^{N\times d}$9) leads to qualitatively different near-critical behaviors in black-hole formation: in five dimensions, a minimal horizon size (mass gap) produces first-order-like transitions (discontinuous collapse), whereas in six dimensions the collapse remains continuously critical with altered exponents (Deppe et al., 2012). This demonstrates how change in dimension and coupling fundamentally reorganizes the collapse properties (interpreted as a "dimensional" arrest of the collapse cascade at the new scale).

6. Quantitative Signatures and Mitigation Strategies in Machine Learning

Empirical and theoretical work in representation learning provides precise criteria and remedies:

• Spectra: Fraction of variance explained, effective rank $\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$0, and singular value decay all encode the degree of collapse. For instance, in contrastive CL, $\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$1 non-zero singular values out of $\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$2 have been observed in practical setups (Chen et al., 2023).
• Loss Formulations: Penalizing feature correlation (FedDecorr), maximizing coding rate (log-det), or enforcing orthogonality directly counteracts collapse (Chen et al., 2023, He et al., 2024, Shi et al., 2022).
• Downstream Impact: Collapse mitigation leads to consistent, sometimes large improvements on standard benchmarks for both classification (federated, collaborative) and transfer learning settings.

Summary table of key areas and signatures:

Domain	Mechanism	Collapse Signature
Nonlinear photonics	Feedback in $\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$3	Collapse of Stokes vector to S₁-axis
Representation learning	Optimization + CL losses	Rank-deficient covariance
Federated learning	Data heterogeneity	Inherited local-global collapse
Riemannian geometry	Gromov–Hausdorff limits	Dim$\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$4dim$\Sigma = \frac{1}{N-1}(Z-\bar{Z})^T (Z-\bar{Z})$5; fibration
Gravitational physics	Dimensionality + extra scale	Discontinuous mass gap (5D), critical scaling (6D)

7. Physical and Algorithmic Implications

Dimensional collapse is not universally deleterious: in systems designed for optimization (e.g., PIMs), the reduction to discrete attractors is the fundamental computational resource. In neural and geometric settings, collapse is typically a failure mode, undermining representational capacity and generalization. In all cases, careful quantitative diagnosis (covariance spectra, scaling laws) and mathematically motivated remedies (orthogonalization, covariance regularization) are essential.

Dimensional collapse, as a concept and observed phenomenon, links diverse scientific disciplines through its unifying theme: the emergent reduction of effective dimensionality in complex high-dimensional systems (Chiavazzo et al., 11 Apr 2025, Jing et al., 2021, Chen et al., 2023, He et al., 2024, Shi et al., 2022, Shaikh et al., 2018, Yamaguchi, 2012, Roos, 2017, Sarwe et al., 2012, Deppe et al., 2012).