Inter-layer Relation Drift in Multilayer Systems

• Inter-layer relation drift is the evolving relationship between layers in complex systems, observed as changes in inter-layer links, coordinated representations, and feature alignments.
• In multilayer networks, it manifests as dynamic switching, structural mismatches, and rewiring that influence synchronization and system stability.
• In deep learning and vision models, drift reflects temporal evolution of representations and cross-scale feature misalignments, impacting learning robustness and performance.

Inter-layer relation drift denotes changes in the relationships that couple one layer to another, but the object that drifts depends on the field in question. In multilayer dynamical networks, it refers to changing inter-layer links, structural mismatch between layers, or statistical evolution of cross-layer degree relations (Eser et al., 2021). In representation learning and continual adaptation, it refers to over-time changes in coordinated layer-wise representations or to the progressive disruption of relationships among layer-wise representations during learning of new tasks (Pashakhanloo et al., 2023). In hierarchical vision models, it refers to misalignment of feature patterns across scales when inter-layer correlations are not explicitly constrained (Yeh et al., 2018).

1. Conceptual scope and operational definitions

The literature operationalizes inter-layer relation drift through different mathematical objects rather than through a single standardized definition. In the cited works, the term is attached to at least three recurrent settings: multilayer network dynamics, deep representation dynamics, and hierarchical feature interactions.

Context	Operational object	Drift form
Multilayer networks	$\xi_i(t)$, $\Delta \mathcal{L}$, $B$	switching, rewiring, or structural mismatch of inter-layer links
Representation learning	$R_{s,t}$, $\mathcal{R}_k$, $r_\ell$	evolving cross-layer similarity structure or perturbation propagation
Hierarchical vision features	$G^{l,m}$, $F_T/F_R$	misalignment or enforced complementarity across scales

In time-switching multilayer synchronization, the binary pattern $\xi_i(t)$ changes while the number of active inter-layer links remains fixed, so drift concerns the identity of coupled node pairs rather than the coupling budget (Eser et al., 2021). In continual learning, the relevant object is the inter-layer relation matrix $R_{s,t}$, whose entries are $\Delta \mathcal{L}$0, and drift is measured by $\Delta \mathcal{L}$1 (Wan et al., 8 May 2026). In pruning-scale perturbation analysis for LLMs, the relevant object is the layerwise evolution of $\Delta \mathcal{L}$2, summarized by the relative reconstruction error $\Delta \mathcal{L}$3 and the absorption coefficient $\Delta \mathcal{L}$4 (Jing et al., 13 Jun 2026). In CNN-based style transfer, drift appears when only per-layer statistics are constrained and the relationships between scales are allowed to vary, which the cross-layer Gram matrix $\Delta \mathcal{L}$5 is designed to prevent (Yeh et al., 2018).

This suggests that the common thread is not any single state variable, but the stability of relations that connect successive or corresponding layers.

2. Dynamical-systems and multilayer-network formulations

In multiplex synchronization, the canonical relation is the inter-layer synchronous manifold $\Delta \mathcal{L}$6 for all $\Delta \mathcal{L}$7. For two identical layers, the manifold $\Delta \mathcal{L}$8 is invariant, and its stability can be analyzed through transverse variational equations and the sign of the maximum transverse Lyapunov exponent (Sevilla-Escoboza et al., 2015). The global inter-layer synchronization error in the FitzHugh–Nagumo switching-link model is

$\Delta \mathcal{L}$9

with $B$0 corresponding to complete inter-layer synchronization (Eser et al., 2021).

When inter-layer links switch in time, the control parameters are the number of active links $B$1 and the switching time $B$2. For static inter-layer topology, $B$3 decreases smoothly and synchronization is achieved only when all node pairs are coupled. Under time-switching links, synchronizability improves, synchronization can occur with much smaller $B$4, and for a critical switching-time the transition becomes abrupt. In the reported example with $B$5 and $B$6, $B$7 remains relatively large for $B$8, then drops abruptly near $B$9–78; the interpretation is a shrinking and eventual disappearance of the basin of attraction of the desynchronized state (Eser et al., 2021). The same work estimates basin sizes probabilistically from $R_{s,t}$0 simulations with random initial conditions and identifies coexistence of a high synchronization error state and a low synchronization error state.

Structural mismatch produces a distinct form of drift. If two multiplex layers differ by a small number of links, $R_{s,t}$1, the exact inter-layer synchronous manifold is no longer invariant. The approximate variational equations acquire the forcing term

$R_{s,t}$2

described as an extra inertial term accounting for structural differences (Leyva et al., 2016). The resulting dynamics does not converge exactly to $R_{s,t}$3; rather, trajectories wander around an inter-layer synchronous configuration, with the drift magnitude controlled by the interplay between stabilizing transverse dynamics and this mismatch-induced forcing. Numerical and experimental results further show a non-trivial relationship between the betweenness centrality of the missing links and the intra-layer coupling strength (Leyva et al., 2016).

A third network-theoretic formulation treats drift as a controlled change from one-to-one to one-to-many inter-layer mapping. In non-identical multilayer networks, symmetric inter-layer coupling corresponds to $R_{s,t}$4, whereas asymmetric coupling randomizes $R_{s,t}$5 while preserving the total number of inter-layer edges. Specific one-to-one inter-layer connections facilitate intra-layer synchronization and make amplitude death appear at lower connectivity strength and frequency mismatch; random one-to-many connections suppress intra-layer homomorphism, degrade remanent synchrony, and require larger coupling and mismatch for amplitude death (Singh et al., 24 Nov 2025). In growing multiplexes, inter-layer degree relations also drift toward a steady-state law. For both preferential and uniform attachment, the conditional mean layer-2 degree obeys

$R_{s,t}$6

even though the full joint degree distributions differ (Fotouhi et al., 2014).

3. Representation manifolds, perturbation propagation, and continual adaptation

In a two-layer linear feedforward network trained online with SGD and weight decay, representational drift is formulated geometrically through a minimum-loss manifold

$R_{s,t}$7

Near $R_{s,t}$8, SGD decomposes into normal and tangential components: normal dynamics follow a multivariate Ornstein–Uhlenbeck process with finite stationary covariance, while tangential motion induces diffusion along the manifold and leaves expected loss unchanged (Pashakhanloo et al., 2023). Hidden representations of eigen-stimuli form an $R_{s,t}$9-dimensional ellipsoid, and drift in that space is a rotational diffusion with pairwise coefficients $\mathcal{R}_k$0. The analysis shows that more frequently presented stimuli drift more slowly, both in fluctuation and in diffusion, and that coordinated changes in $\mathcal{R}_k$1 and $\mathcal{R}_k$2 preserve the input–output map to first order (Pashakhanloo et al., 2023).

In continual learning, the same issue is posed directly as inter-layer relation drift. For a sample $\mathcal{R}_k$3, the inter-layer relation matrix under model $\mathcal{R}_k$4 is

$\mathcal{R}_k$5

and drift is measured by $\mathcal{R}_k$6 (Wan et al., 8 May 2026). Theoretical analysis connects forgetting to classification margin shrinkage and then bounds that shrinkage by two terms: the norm of summed residual deviations $\mathcal{R}_k$7, and a term proportional to inter-layer relation drift. The main bound contains

$\mathcal{R}_k$8

so increasing drift directly enlarges the forgetting bound (Wan et al., 8 May 2026). SR$\mathcal{R}_k$9-LoRA addresses this by constructing relation matrices from the previous and current models on current-task samples, computing their SVDs, and aligning the corresponding singular values with a Smooth L1 loss. The robustness argument is spectral: $r_\ell$0 The reported effect is that catastrophic forgetting is mitigated, with the advantage becoming more pronounced as the number of tasks increases (Wan et al., 8 May 2026).

For pruning-scale perturbations in transformer LLMs, the relevant question is how a perturbation injected at one layer propagates through later layers. With

$r_\ell$1

the empirical result is strongly layer- and scale-dependent: early layers amplify perturbations, while middle and late layers absorb them at pruning scale, with relative L2 drift decreasing monotonically across depth and direction realigning toward the unperturbed hidden-state trajectory (Jing et al., 13 Jun 2026). Under small perturbations such as $r_\ell$2, amplification is observed across all layers; absorption emerges smoothly only as perturbation magnitude grows to pruning scale such as $r_\ell$3. The absorption coefficient

$r_\ell$4

is then used to correct layer-wise sparsity profiles, improving OWL and AlphaPruning by reducing perplexity by $r_\ell$5 and boosting zero-shot accuracy by $r_\ell$6 across multiple model families at $r_\ell$7 sparsity (Jing et al., 13 Jun 2026).

4. Hierarchical feature coherence in vision models

In style transfer, inter-layer relation drift is the misalignment of feature patterns across scales that arises when each layer is constrained independently. Standard within-layer style losses match Gram matrices

$r_\ell$8

but do not constrain how small-scale and large-scale features co-occur spatially. The cross-layer alternative uses

$r_\ell$9

which correlates activations at different depths at corresponding spatial positions (Yeh et al., 2018). The paper reports that cross-layer Gram matrices are sufficient to control within-layer statistics and give visible improvements on hard styles with long-range or hierarchical structure. In the pairwise descending configuration, the parameter count is $G^{l,m}$0 for cross-layer constraints versus $G^{l,m}$1 for within-layer constraints, yet the cross-layer representation is more informative for preserving patterns such as dots arranged on curves, large black areas, relief strokes, organized color blocks, and long curves (Yeh et al., 2018). The same work also reports that multiplicative loss $G^{l,m}$2 emphasizes strong content boundaries and eliminates the explicit balancing hyperparameter $G^{l,m}$3.

A related but explicitly constructive formulation appears in single-image reflection removal. There, inter-layer complementarity is modeled through

$G^{l,m}$4

with a dual-stream architecture producing $G^{l,m}$5 and a high-frequency block producing $G^{l,m}$6 (Huang et al., 19 May 2025). The low-frequency residual is tied to $G^{l,m}$7, the high-frequency residual is defined as the remaining component, and gradient exclusion is enforced through

$G^{l,m}$8

The inter-layer complementarity attention mechanism cross-reorganizes channels by splitting

$G^{l,m}$9

forming

$F_T/F_R$0

and then applying intra-flow self-attention and cross-flow attention before restoring the two streams (Huang et al., 19 May 2025). This is an explicit anti-drift design: transmission and residual streams are repeatedly re-synchronized at the feature level. The reported complexity comparison for LCAB versus DAIB is $F_T/F_R$1M versus $F_T/F_R$2M parameters, $F_T/F_R$3G versus $F_T/F_R$4G FLOPs, and $F_T/F_R$5ms versus $F_T/F_R$6ms speed (Huang et al., 19 May 2025).

5. Architecture search, operator theory, and abstract control of drift

In differentiable neural architecture search, standard DARTS-style methods optimize the architecture weights on each edge independently. ITNAS replaces that independence assumption with an explicit inter-layer transition mechanism. Outer edges are optimized independently, but inner-edge architecture probabilities are derived from predecessor edges by

$F_T/F_R$7

where each $F_T/F_R$8 is a row-stochastic transition matrix and $F_T/F_R$9 are attention weights over predecessors (Ma et al., 2020). The resulting search problem is a sequential decision process on the DAG, and the transition-induced iterative edge pruning procedure updates descendants after pruning predecessors rather than treating edges as isolated decisions. On CIFAR-10, the reported error is $\xi_i(t)$0; on ImageNet, the reported Top-1 error is $\xi_i(t)$1 (Ma et al., 2020). The architectural point is that uncontrolled drift of operation distributions across connected edges is replaced by learned, attention-weighted probability transitions.

An abstract operator-theoretic framework pushes the same idea to the level of general deep compositions. Drift maps $\xi_i(t)$2 satisfy $\xi_i(t)$3, anchor sets $\xi_i(t)$4 are enforced by metric projections $\xi_i(t)$5, and event blocks are

$\xi_i(t)$6

with effective block factor

$\xi_i(t)$7

The drift–projection convergence theorem states that if $\xi_i(t)$8, then the iterates converge to the anchor point $\xi_i(t)$9 (Alpay et al., 13 Aug 2025). The same work proves that nested affine anchors with singleton intersection imply strong convergence to a unique limit, that softmax is $R_{s,t}$0-Lipschitz in $R_{s,t}$1, and that attention layers are contractive under sufficient conditions such as $R_{s,t}$2 or, in the equal-$R_{s,t}$3 case, $R_{s,t}$4 (Alpay et al., 13 Aug 2025). In this abstraction, inter-layer relation drift is the evolution of pairwise distances and anchor-relative geometry under products of Lipschitz maps.

6. Recurrent themes, common misconceptions, and limitations

A recurrent misconception is that drift is necessarily equivalent to error or disorder. The cited literature does not support that simplification. In the two-layer linear network, drift is a diffusion along a minimum-loss manifold and can coexist with stable task performance (Pashakhanloo et al., 2023). In switching multilayer synchronization, time-varying inter-layer links can improve synchronizability and reduce the connectivity required for synchronization, even though they also create bistability and abrupt transitions (Eser et al., 2021). In non-identical multilayer networks, asymmetric one-to-many inter-layer connectivity suppresses amplitude death at low coupling and mismatch, even while degrading intra-layer synchronization and memory (Singh et al., 24 Nov 2025).

A second recurrent theme is that purely local descriptions are often inadequate. Within-layer Gram statistics do not preserve cross-scale organization in style transfer (Yeh et al., 2018). Local layer-importance signals do not capture downstream perturbation absorption in LLM pruning (Jing et al., 13 Jun 2026). Independent edge-wise architecture weights ignore DAG-induced dependence in NAS (Ma et al., 2020). These results, taken together, support the view that cross-layer relations are first-class dynamical or representational objects, not merely by-products of per-layer quantities.

The literature also delineates clear limitations. The switching FitzHugh–Nagumo study is predominantly numerical and does not use formal tools like the Master Stability Function, Floquet theory, or explicit eigenvalue conditions (Eser et al., 2021). The non-identical multiplex analysis is an approximate treatment whose usefulness is strongest for small or moderate topological differences (Leyva et al., 2016). The linear-network theory is deliberately restricted to a strictly linear, two-layer, feedforward architecture with stationary input distribution (Pashakhanloo et al., 2023). The non-identical multilayer trade-off study considers only $R_{s,t}$5–$R_{s,t}$6 layers and treats rewiring as static rather than adaptive in time (Singh et al., 24 Nov 2025).

This suggests that inter-layer relation drift is best understood as a family of structurally related phenomena: time-varying or mismatched inter-layer couplings in multilayer dynamics, perturbation-sensitive evolution of layerwise representations in deep models, and cross-scale coherence or incoherence in hierarchical feature systems. Across these settings, the decisive question is whether the mechanisms that couple layers amplify deviations, absorb them, or constrain them to a stable manifold.