Layer Collapse in Layered Systems

• Layer-collapse property is a structural regime in layered systems where successive layers lose independent organization, reducing representational diversity and specific functionality.
• It is diagnosed through layer-indexed observables such as similarity metrics, covariance structures, and decision transitions, which vary across neural, physical, and network domains.
• Understanding layer collapse informs strategies for model pruning, compression, and stability analysis, impacting transformer LLMs, multiplex networks, and condensed-matter systems.

The layer-collapse property denotes a family of phenomena in layered systems in which the functional distinctness, representational diversity, or physical integrity of layers is lost, either spontaneously or by design. In recent arXiv literature, the term is used for late-layer degradation of logical consistency in transformer LLMs, early-layer redundancy driven by a persistent super-outlier in diffusion LLMs, structured removal or merging of layers in neural compression, variability collapse of last-layer features in supervised learning, discontinuous failure in multiplex networks, and the disappearance of ordered interlayer or interfacial states in condensed-matter and fluid systems (Miyashita et al., 13 May 2026, Conzelmann et al., 7 May 2026, Quétu et al., 2024, Costa et al., 2021, Stahl et al., 2019, Harvey et al., 2023). Taken together, these usages suggest that “layer collapse” is not a single mechanism but a recurring structural regime: successive layers cease to contribute independent organization, or an interlayer order parameter vanishes.

1. Scope and operational meanings

Across the cited literatures, “collapse” is always diagnosed through a layer-indexed observable rather than by nomenclature alone. In machine learning, the collapsed object is typically a hidden-state geometry, an ontology-preserving algebraic structure, a decision trajectory, or a feature covariance structure. In physical and networked systems, the collapsed object is an interlayer dimer state, a vapor film, a weakly percolating multilayer component, or a bound skyrmion pair. The operational content therefore depends on what each field regards as the layer’s essential function.

Domain	What collapses	Diagnostic signature
Transformer LLMs	Logical/algebraic consistency	Final-layer accuracy drop; negative or near-zero structural metric
Diffusion LMs	Distinctness of early hidden layers	$\mathrm{sim}(i,j)>0.98$ and persistent super-outlier
Neural compression	Intermediate functional contribution	Near-identity layers removable with little loss
Neural collapse literature	Within-class or target-subspace variability	NC or NRC metrics tending to zero
Multiplex/physical layers	Interlayer connectivity or ordered phase	Discontinuous failure, $\Delta \to 0$, or instability threshold

This breadth matters because the same phrase can refer to sharply different causal stories. In some cases, collapse is a failure mode that reduces logical consistency or accuracy; in others, it is a regularity that enables exact or near-exact compression. A plausible implication is that the most useful general definition is operational: a layered system has entered a collapse regime when one can no longer attribute independent structure, transformation, or stability to the affected layers.

2. Logical and representational collapse in LLMs

In "Controlling Logical Collapse in LLMs via Algebraic Ontology Projection over F2," the layer-collapse property is formulated as Late-layer Collapse: when a transformer is inspected layer by layer, its ability to represent formally consistent logical relations improves in early and middle layers and then degrades in the final layers (Miyashita et al., 13 May 2026). The paper introduces Algebraic Ontology Projection (AOP), which maps hidden states $\mathbf h \in \mathbb R^d$ into $\mathbb F_2^n$ through

$$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$

with $\gamma=4$, and interprets bitwise AND as intersection. The core algebraic constraint is the Liskov Substitution Principle for inheritance,

$$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$

with analogous constraints for part-whole and negation. AOP is trained only on 42 relational keys: 15 is-a pairs, 12 has-a pairs, and 15 negation pairs. The resulting structure is strongly layer-dependent. To quantify it, the paper defines Semantic Crystallisation (SC) via

$$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$

with $SC(L)\approx[-1,1]$. Positive $SC$ indicates a crystalline layer, $\Delta \to 0$0 an unstructured gas phase, and $\Delta \to 0$1 a collapsed or melting phase. Empirically, the maximal-$\Delta \to 0$2 layer coincides with maximal zero-shot inclusion accuracy in every condition. Reported peak results include 93.33% for Gemma-2 Instruct with prompt at layer 14 ($\Delta \to 0$3), 86.67% for Qwen2.5 Instruct with prompt at layer 11 ($\Delta \to 0$4), and 86.67% for mpnet with prompt only at layer 3 ($\Delta \to 0$5). In 7 of 10 model/prompt conditions, inclusion accuracy falls by more than 10 percentage points in the final 5 layers relative to the peak. The same study further argues that a short “expert taxonomic hierarchy” system prompt acts as an algebraic boundary condition, and that only the combination of instruction-tuned weights and a structured system prompt fully prevents Late-layer Collapse.

A different but related usage appears in "Layer Collapse in Diffusion LLMs," which analyzes LLaDA-8B and identifies a collapse regime in which early layers exhibit highly similar activations dominated by a single persistent super-outlier (Conzelmann et al., 7 May 2026). Hidden-state redundancy is measured by

$\Delta \to 0$6

Layers are treated as collapsed when $\Delta \to 0$7; in LLaDA-8B, blocks of 10–15 layers reach $\Delta \to 0$8. The super-outlier’s average magnitude can be up to five times larger than the runner-up and persists at the same channel index across many layers. Despite the apparent redundancy, pruning that single top-magnitude channel, index 3848, on a GSM8K subset collapses LLaDA-8B accuracy from approximately 83% to 0%, producing repetitive loops, whereas pruning the top channel in Llama-3.1-8B reduces accuracy by 4%. The paper explicitly rejects an undertraining explanation: early DLM layers show lower Hill-estimator tail exponents $\Delta \to 0$9, which the authors interpret as an overtraining signature.

These two LLM results define collapse differently—late logical melting versus early redundancy around a dominant direction—but they converge on the same structural lesson. Layer quality is not monotone in depth, and the final or deepest representation need not be the most faithful one. This suggests that layer selection, prompting, pruning, and even deployment-time decoding should be conditioned on internal layer dynamics rather than on the terminal layer by default.

3. Collapse as a compression primitive

A third line of work uses layer collapse constructively. "Understanding Performance Collapse in Layer-Pruned LLMs via Decision Representation Transitions" studies multiple-choice LLMs through Decision Margin

$\mathbf h \in \mathbb R^d$0

and Option Frequency

$\mathbf h \in \mathbb R^d$1

The paper reports a two-phase structure: a Silent Phase, in which $\mathbf h \in \mathbb R^d$2 and predictions remain biased toward a spurious option, and a Decisive Phase, beginning at the transition point

$\mathbf h \in \mathbb R^d$3

Pruning layers in the Decisive Phase reduces margins but preserves the ability to cross $\mathbf h \in \mathbb R^d$4, whereas pruning any Silent-Phase layer pushes the transition beyond the new network depth, keeps $\mathbf h \in \mathbb R^d$5 throughout, and collapses accuracy to chance (Shi et al., 8 May 2026). The transition position strongly predicts the maximum prune rate before collapse, with Pearson $\mathbf h \in \mathbb R^d$6 across models and tasks. In this sense, layer collapse is not only representational; it can be recast as the failure of a depth-indexed decision transition.

"LLM Pruning via Layer Collapse" operationalizes collapse as a mergeability property of adjacent transformer layers (Yang et al., 2024). Its Reserving-Differences-while-Seeking-Common merge defines

$\mathbf h \in \mathbb R^d$7

after which layers $\mathbf h \in \mathbb R^d$8 are removed. The motivation is explicitly layer-wise similarity. For 20 random Wikipedia sentences, cosine similarities $\mathbf h \in \mathbb R^d$9 are reported as extremely close to 1.0 for layers $\mathbb F_2^n$0, and after collapsing a four-layer block the minimum cosine similarity over 4096 dimensions remains above 0.996. At approximately 25–27% pruning, the method retains over 80% average task performance: Llama2-7B goes from 46.55 to 37.46 average score when pruned from 32 to 23 layers, and Llama2-13B from 55.50 to 47.55 when pruned from 40 to 30 layers. The same paper states that 40→30 layer collapse yields a 25% reduction in sequential transformer blocks, with corresponding compute savings, and that recovery is possible with limited post-training.

LaCoOT extends this constructive notion to vision architectures through optimal transport regularization (Quétu et al., 2024). Given per-block input and output empirical measures $\mathbb F_2^n$1 and $\mathbb F_2^n$2, it penalizes

$\mathbb F_2^n$3

and adds the average block-wise regularizer to the task loss. A central lemma states that if $\mathbb F_2^n$4, then the block acts as the identity on the support of $\mathbb F_2^n$5. The corresponding post-training algorithm collapses the layer with minimal empirical transport distance and requires no fine-tuning. On CIFAR-10 with ResNet-18, the original model has 91.77 top-1 accuracy and 140.2M MACs; LaCoOT with $\mathbb F_2^n$6 reaches 90.99 top-1 with 64.7M MACs, and with $\mathbb F_2^n$7 reaches 89.01 top-1 with the same MAC count. Here collapse is neither failure nor pathology; it is a provable near-identity criterion for depth reduction.

4. Collapse in supervised representation geometry

In the neural-collapse literature, the term refers to the terminal simplification of last-layer geometry. "A Geometric Analysis of Neural Collapse with Unconstrained Features" formalizes the classical classification setting: class means and classifier weights collapse to a Simplex Equiangular Tight Frame, while within-class covariance

$\mathbb F_2^n$8

tends to zero (Zhu et al., 2021). Under the unconstrained feature model with cross-entropy and weight decay, the paper proves that global minimizers exhibit exactly this ETF structure and that all other critical points are strict saddles. The result turns “collapse” from a heuristic observation into a global landscape statement.

"Quantifying the Variability Collapse of Neural Networks" isolates the NC1 component through the Variability Collapse Index

$\mathbb F_2^n$9

where $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$0 (Xu et al., 2023). The paper shows that minimizing VCI is equivalent to minimizing the linear-probing mean-squared error optimum, proves invariance under invertible linear transformations, and argues that $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$1 is numerically stable because $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$2 has a much larger spectral gap than $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$3 alone. Across pretraining sweeps, VCI remains strongly positively correlated with mean log-odds gain, with Pearson $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$4 for varying softmax temperature $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$5 and $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$6 for varying cosine-similarity regularizer strength $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$7.

Layerwise robustness complicates this otherwise terminal picture. In "On the Robustness of Neural Collapse and the Neural Collapse of Robustness," Su et al. track NC1, NC2, and NC4 across hidden layers under standard training and adversarial training (Su et al., 2023). Under standard training, clean data show the expected cascading collapse toward deeper layers, but adversarially perturbed inputs preserve reliable simplex structure only in the first few layers; later layers cease to collapse in the NC sense. On CIFAR-10, $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$8 on adversarial data decreases from approximately 1.0 at $$\mathbf z=\sigma\!\Bigl(\gamma W_2\tanh(W_1\mathbf h-\boldsymbol\theta)\Bigr),\qquad \hat z_i=\begin{cases} 1,& z_i\ge 0.5\ 0,& z_i<0.5 \end{cases}$$9 to approximately 0.2 at $\gamma=4$0 and then saturates around 0.2–0.3, whereas the clean curve falls below 0.05 by the final layer. Under adversarial training, clean and perturbed curves both decrease across layers and fall below 0.05 by the end. The same paper reports “cluster-leaping”: adversarial examples in standard-training networks jump between simplex vertices rather than merely dispersing.

"The Prevalence of Neural Collapse in Neural Multivariate Regression" broadens the phenomenon to continuous-output tasks by introducing Neural Regression Collapse (NRC) (Andriopoulos et al., 2024). NRC1 states that normalized last-layer features collapse onto the subspace spanned by the top-$\gamma=4$1 principal components, where $\gamma=4$2 is the target dimension; NRC2 aligns this subspace with $\gamma=4$3; NRC3 constrains the normalized last-layer weights to the functional form $\gamma=4$4. Under the unconstrained feature model with strictly positive regularization, the paper derives these structures as global minima; with zero regularization, it states that there is no collapse. This suggests that collapse in representation geometry is not unique to classification but can emerge whenever regularization selects a low-complexity terminal structure.

5. Collapse transitions in networks and physical layered media

Outside machine learning, the layer-collapse property often denotes a discontinuous transition in the actual layered object. In weak multiplex percolation, da Costa et al. study an $\gamma=4$5-layer network in which each node must have at least one neighbor in every layer (Costa et al., 2021). For symmetric, uncorrelated layers with $\gamma=4$6, the self-consistency equation

$\gamma=4$7

has the full-percolation fixed point $\gamma=4$8, but its stability is governed by $\gamma=4$9, where $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$0. The critical point is therefore

$$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$1

For $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$2, infinitesimal damage $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$3 can trigger a discontinuous jump of the giant weakly percolating component from $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$4 to $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$5 at $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$6, with no square-root singularity. Above threshold, collapse time scales as $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$7; below it, relaxation time scales as $$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$8.

In layered quantum matter, Stahl et al. identify the collapse of inter-layer molecular-orbital dimers in photo-excited 1T-TaS$$A\subseteq B \Longleftrightarrow \mathbf a \odot \mathbf b=\mathbf a,$$9 (Stahl et al., 2019). The insulating commensurate CDW phase is characterized by a dimerization order parameter $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$0, with $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$1 Å in the CCDW and $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$2 in the hidden CDW state. X-ray diffraction shows that the CCDW double peak at $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$3 collapses and a sharp peak at $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$4 reappears in the photo-induced state. The reported pump parameters are $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$5 nm, $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$6 ps, and $$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$7 mJ/cm$$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$8; the characteristic collapse time is bounded above by the pulse duration and consistent with earlier 400–600 fs amplitude-mode melting. The same account links dimer breaking to the collapse of an interlayer bonding–antibonding gap of approximately 200–300 meV and the onset of metallic conduction.

Fluid-mechanical and granular examples are likewise phrased as layer collapse. In the Leidenfrost problem, Harvey and Burton model the vapor film beneath a liquid on a hot solid and identify collapse as the failure of the film during cooling (Harvey et al., 2023). Their simulations attribute the instability to vapor inertia, ordinarily neglected in lubrication theory. Balancing Bernoulli suction against Laplace pressure yields a critical wavelength

$$q(L)=\frac{\mathcal L_{alg}(L)}{\rho(L)^2},\qquad SC(L)=\bigl(\mu_{rand}-q(L)\bigr)\sigma^2_{rand},$$9

and collapse occurs when perturbations below this scale grow until the interface touches the solid. The reported local failure temperature is typically approximately $SC(L)\approx[-1,1]$0C on smooth surfaces. In inclined wet granular layers, impact-induced collapse is modeled by a block-slip criterion,

$SC(L)\approx[-1,1]$1

equivalently

$SC(L)\approx[-1,1]$2

with DEM-informed attenuation factor $SC(L)\approx[-1,1]$3 (Takizawa et al., 2018). Collapse appears only when the inclination is close to the maximum stable angle and the base acceleration is sufficiently large.

A magnetic example appears in "Skyrmions in Synthetic Antiferromagnets: Collapse and Nucleation," where collapse is explicitly layer-sequential rather than simultaneous (Potkina, 5 Jun 2026). Minimum-energy-path calculations in a two-layer reduced lattice model show a first saddle for lower-layer collapse and a second for upper-layer collapse, with a possible single-layer skyrmion intermediate. The main saddle energy $SC(L)\approx[-1,1]$4–$SC(L)\approx[-1,1]$5 changes only weakly with lateral size, while the bound-pair minimum decreases from about $SC(L)\approx[-1,1]$6 at $SC(L)\approx[-1,1]$7 to $SC(L)\approx[-1,1]$8 at $SC(L)\approx[-1,1]$9 because of a boundary penalty. The reverse nucleation barrier is much larger than the collapse barrier, which the authors interpret as consistent with assisted layer-sequential writing.

6. Recurrent motifs, misconceptions, and research significance

Several motifs recur across these otherwise heterogeneous literatures. First, collapse is usually layer-resolved: AOP uses per-layer algebraic loss and SC; DLM analysis uses pairwise layer similarity and outlier statistics; pruning studies use layerwise decision formation or feature transport; NC studies compute covariance-based metrics per depth; multiplex and physical systems define control parameters tied to interlayer stability. Second, collapse often appears as a threshold phenomenon. Examples include $SC$0, $SC$1, the decision transition at $SC$2, $SC$3, the Leidenfrost wavelength threshold $SC$4, and the wet-granular slip boundary (Miyashita et al., 13 May 2026, Conzelmann et al., 7 May 2026, Shi et al., 8 May 2026, Costa et al., 2021, Harvey et al., 2023, Takizawa et al., 2018).

A common misconception is that collapse is always a sign of undertraining or always occurs in the deepest layers. The DLM study explicitly argues the reverse: early-layer collapse in diffusion LLMs is associated with overtraining and a persistent indispensable super-outlier (Conzelmann et al., 7 May 2026). Another misconception is that collapse is uniformly harmful. In AOP and decision-transition analyses, collapse denotes loss of logical consistency or decision capability; in LaCo and LaCoOT, closely related structural redundancy is the condition that makes aggressive pruning possible (Yang et al., 2024, Quétu et al., 2024). The term therefore names a structural state, not a value judgment.

The research significance of the layer-collapse property lies in its diagnostic and design roles. In LLM reasoning, SC provides a gradient-free criterion for selecting layers without held-out data, and the same work proposes prompt design through token-level SC analysis and future training with algebraic constraint losses (Miyashita et al., 13 May 2026). In representation learning, VCI offers an invariant and numerically stable proxy for variability collapse and transferability, while adversarial NC analysis shows that robustness is partly a question of how collapse propagates through depth (Xu et al., 2023, Su et al., 2023). In compression, constructive collapse supplies principled removal rules rather than heuristic depth truncation (Yang et al., 2024, Quétu et al., 2024). In physical and networked systems, collapse thresholds identify the onset of catastrophic transitions and the asymmetry between failure and recovery (Costa et al., 2021, Stahl et al., 2019, Potkina, 5 Jun 2026).

Taken together, these results support a broad but precise interpretation. The layer-collapse property is a depth- or stacking-dependent loss of independent structure, diagnosed by task-specific invariants, order parameters, or stability criteria. Its consequences range from hallucination risk and pruning failure to exact depth reduction, discontinuous percolation collapse, phase switching, and interfacial instability. The unifying research problem is therefore not whether collapse exists, but which quantity collapses, where the transition occurs, what control parameter governs it, and whether the phenomenon should be prevented, exploited, or engineered.