---
title: Neural Jamming Phase Diagram
url: https://www.emergentmind.com/topics/neural-jamming-phase-diagram
type: topic
---

# Neural Jamming Phase Diagram

A neural jamming phase diagram delineates the critical boundaries and regimes that emerge when high-dimensional learning systems, such as fully connected deep neural networks or constraint satisfaction models like the perceptron, encounter a transition between phases in which all training constraints can be satisfied and phases where a finite fraction remain unsatisfied after optimization. This transition exhibits direct analogies with the physical jamming of disordered repulsive particles, manifesting in singular loss landscape features, power-law distributions, and rich critical phenomena. Contemporary research has extended these ideas to the domains of deep learning, frictional particle suspensions, and even the emergence of large-scale cognitive phenomena in artificial neural architectures, providing a unifying framework across statistical physics and machine learning.

## 1. Core Control Parameters and Critical Criteria

Fundamental to the neural jamming paradigm are intensive control parameters that govern the transition:

- **In fully connected networks** [1809.09349], control parameters are:
    - $N$ = total number of effective trainable parameters (typically the raw parameter count for well-initialized, deep, fully connected ReLU nets).
    - $P$ = number of training examples.
    - $\alpha = P/N$ ("load" or "density").

- **Perceptron and other CSPs** [1501.03397, 1607.00966, 1902.08243]:
    - $\alpha = M/N$ (constraint density), where $M$ is the number of constraints.
    - Margin parameter $\kappa$ or $\sigma$ defines the gap in the constraint geometry and separates convex from non-convex regimes.

- **Physical analogs** [2502.18743, 2507.08197]:
    - Packing fraction $\phi$ (density of constituents or learned representations).
    - Shear stress $\sigma$ or $\Sigma$ (external perturbations or noise).
    - Effective temperature $T_n$ (computational "cooling," e.g., inverse compute budget).

**Jamming Threshold**: For neural nets, a critical load $\alpha^*$ is observed, such that for $\alpha < \alpha^*$, the minimization achieves zero training loss ("SAT" or over-parameterized), and for $\alpha > \alpha^*$, a finite fraction of constraints remain unsatisfied ("UNSAT" or under-parameterized). This threshold can be empirically determined, and analytic bounds exist given landscape curvature properties, e.g., $N^* \le P / C_0$ for fully connected nets where $C_0$ is the limiting fraction of negative modes in the Hessian [1809.09349].

## 2. Structure and Features of the Neural Jamming Phase Diagram

A diagrammatic representation of jamming transitions organizes distinct phases in the parameter space:

- **Diagrams in neural network learning** [1809.09349, 2012.15110]:
    - Axes: horizontal—$\alpha = P/N$ or width $h$; vertical—final training loss $\mathcal{L}_{final}$.
    - Phases:
        - **SAT ($\alpha < \alpha^*$ or $h > h^*$):** All constraints (examples) can be fitted; loss can reach zero. Accompanied by many flat directions in parameter space and marginally stable landscape regions.
        - **UNSAT ($\alpha > \alpha^*$ or $h < h^*$):** Some constraints remain unsatisfied; nonzero final training loss; a finite fraction of data is not fit.
    - The SAT/UNSAT boundary exhibits properties akin to the jamming transition in particulate matter.

- **Perceptron and constraint satisfaction models** [1501.03397, 1607.00966, 1902.08243]:
    - Phase diagrams in the $(\alpha,\kappa)$ or $(\alpha,\sigma)$ plane, with SAT/UNSAT (unjammed/jammed) boundaries given by analytical expressions, e.g.,
        $$
        \alpha_J(\sigma) = \left[ \int_\sigma^\infty Du\,(u-\sigma)^2 \right]^{-1}
        $$
    - For linear/hinge losses, critical behavior (jamming-like singularities) pervades the entire "Gardner/RSB" region beyond the loss of convexity.

- **Extended phase diagrams in physical/simulation settings** [2502.18743]:
    - High-dimensional surfaces $\,\phi_J(\sigma, \mu_s, \mu_r)$, where $\phi_J$ is the critical packing fraction defining jamming, and the phase behavior is sampled over friction, stress, and rolling friction.

## 3. Analogy to Physical Jamming and Universality Classes

The jamming transition in neural networks is an instantiation of a broader universality observed in disordered statistical systems [1501.03397, 1607.00966, 2012.15110, 2507.08197]:

- **Mapping of variables:**

| Physical Jamming                       | Neural Jamming                                    |
|----------------------------------------|---------------------------------------------------|
| Particle positions $r_i$ (DoF $=N$)    | Network weights $W$ (DoF $=N$)                   |
| Pair overlaps $\Delta_{ij}=2R-|r_i-r_j|$  | Data “gaps” $\Delta_\mu = \epsilon - y_\mu f(x_\mu;W)$ |
| Density $\phi$                         | Load $\alpha = P/N$                              |
| Force and gap laws (e.g., isostatic)   | Fraction of unsatisfied constraints $N_\Delta/N$  |
| Fluid/solid phases                     | SAT/UNSAT (fit/unfit)                             |

- **Universality and criticality**: Non-convex CSPs (e.g., perceptron for $\kappa<0$) and deep nets with nonsmooth activation landscapes display critical behavior—power-law gap and force distributions, hierarchical landscape structure, and Gardner marginality—associated with full replica symmetry breaking (RSB). By contrast, convex regimes are hypostatic and non-critical [1501.03397, 1902.08243].

## 4. Critical Exponents, Scaling Laws, and Landscape Properties

At the jamming threshold, neural and statistical models exhibit singular statistical features:

- **Power-law distributions**:
    - For data "gaps" or overlaps $\Delta$ in hinge-like settings [1809.09349, 2012.15110, 1501.03397, 1902.08243]:
        $$
        P_+(\Delta) \sim \Delta^\theta \quad (\Delta > 0),\qquad P_-(\Delta) \sim (-\Delta)^{-\gamma} \quad (\Delta < 0)
        $$
        Typical exponents:
        - Fully connected ReLU: $\theta\approx0.3$, $\gamma\approx0.2$ [1809.09349]
        - Tanh: $\theta\approx0.2$, $\gamma\approx0.16$
        - Spherical perceptron, hard spheres: $\theta\approx0.4231$, $\gamma\approx0.4127$ [1501.03397, 1902.08243]
    - The density of states (eigenfrequencies) of the landscape Hessian exhibits a delta-peak at zero, a gap, and a continuous part—signifying an extensive number of flat directions, as in hypostatic (ellipsoidal) jamming [1809.09349, 2012.15110].

- **Marginal stability and Gardner phase**:
    - In the "UNSAT-RSB" critical jammed regime, criticality persists throughout the region, not just at a single transition [1902.08243].
    - The landscape acquires a hierarchy of minima leading to avalanche-like optimization dynamics: optimization proceeds via abrupt changes in the set of learned patterns [1809.09349].

- **Scaling of observables** [1607.00966, 2012.15110]:
    - Near the jamming point the smallest gap scales as $h_{min}\sim N^{-1/(1-\gamma)}$, the smallest nonzero force as $f_{min}\sim N^{-1/(1+\theta)}$.
    - Gap and force exponents saturate theoretical bounds, linking learning models to infinite-dimensional sphere packings.

## 5. Generalizations: Extended Phase Spaces and Neural Surrogates

Recent work extends neural jamming to:

- **Physical suspensions and neural surrogates** [2502.18743]:
    - Deep graph-convolutional networks (DeepGCN) predict jamming in particulate suspensions by learning frictional contact networks as a function of packing fraction, stress, sliding and rolling friction.
    - Critical surface $\phi_J(\sigma,\mu_s,\mu_r)$ separates fluid and jammed states, enabling rapid surrogate modeling and data-driven materials design.

- **Abstract neural/thermodynamic phase diagrams** [2507.08197]:
    - A neural phase diagram is defined over effective temperature ($T_n\propto1/{\rm compute}$), density of representations ($\phi_n$), and stress ($\Sigma_n$ from data/parameter noise).
    - Emergent "consciousness" or global information integration is predicted to arise when parameters intersect a critical jamming surface $F(T_n,\phi_n,\Sigma_n)=0$.
    - Physical and neural exponents (e.g., correlation-length $\nu\approx0.7$) are conjectured to coincide.

- **Deep learning regime decomposition** [2012.15110]:
    - Distinguishes three learning phases in the $(h,\alpha)$ plane:
        1. Under-parameterized ("jammed"; frequent bad minima, non-zero training loss)
        2. Over-parameterized/NTK ("lazy training"; kernel-dominated, minimal feature evolution)
        3. Over-parameterized/feature learning (dynamical kernel, efficient invariant learning)
    - The boundary separating these regimes is governed by the jamming transition, and properties of the landscape Morse structure.

## 6. Open Questions and Implications

The neural jamming framework continues to raise foundational questions:

- **Double descent and data structure**: Why performance improves beyond jamming and under what data symmetries learning can evade the curse of dimensionality [2012.15110].
- **Architectural optimization**: How non-convexity, stochastic noise, and neural architecture (e.g., CNNs, LLMs) sculpt phase boundaries and optimize generalization [2012.15110, 2507.08197].
- **Physicality of learning**: The border between statistical and physical jamming exponents, universality across activation/loss types, and Gardner marginality in real-world data settings remain active research frontiers [1501.03397, 2012.15110, 2507.08197].
- **Materials design applications**: Rapid, GNN-driven exploration of jamming in particulate and suspension-based materials has direct translational implications for engineering design [2502.18743].

## 7. Summary Table of Key Regimes and Exponents

| Regime                      | Curvature/Isostaticity     | Critical Exponents $[\theta, \gamma]$         | Comments                                 |
|-----------------------------|----------------------------|-----------------------------------------------|------------------------------------------|
| Convex SAT (perceptron)     | Hypostatic ($z<1$)         | No power laws                                 | Landscape smooth, RS                     |
| Non-convex SAT/UNSAT (RSB)  | Isostatic ($z=1$)          | $\approx(0.42, 0.41)$                         | Gap/force power laws, RSB, marginality   |
| Deep FC ReLU jamming        | Hypostatic ($N_\Delta/N<1$)| $\approx(0.3, 0.2)$ (ReLU, random)            | Hierarchical landscape, avalanches       |
| Suspension GNN jamming      | Isostatic threshold $Z_{iso}$| —                                           | Neural model predicts $\phi_J$ surface   |

Exploration of the neural jamming phase diagram continues to advance our understanding of emergent criticality, both as a physical and computational phenomenon, bridging statistical mechanics with learning theory across domains [1809.09349, 1501.03397, 2502.18743, 2012.15110, 1607.00966, 1902.08243, 2507.08197].

Source: https://www.emergentmind.com/topics/neural-jamming-phase-diagram