Cross-Time Nonlinear Rank in Dynamic Systems

• Cross-Time Nonlinear Rank is a family of localized rank notions that quantify effective dimensions in nonlinear systems as they evolve over time, data size, or depth.
• It underpins phase transitions in model stability and observability, impacting overparameterized learning, low-rank manifold adaptations, and time-augmented control conditions.
• The concept informs adaptive strategies in deep networks and nonparametric analyses of temporal data, highlighting shifts in rank due to shocks, bifurcations, and noise.

Searching arXiv for recent and relevant papers on nonlinear rank notions across time, dynamics, and low-rank manifolds. “Cross-Time Nonlinear Rank” (Editor’s term) can be used as an umbrella designation for rank notions in which nonlinear structure is indexed by training data size, explicit time, network depth, or temporal ordering, and in which qualitative behavior changes when a rank threshold is crossed. In current work, this includes nonlinear model rank and linear stability thresholds in overparameterized learning, rank conditions on augmented time-state manifolds for observability and controllability, rank-adaptive evolution on low-rank matrix and tensor manifolds, depth-dependent nonlinear rank in homogeneous networks, and rank trajectories or rank-order transforms for temporal data (Zhang et al., 2022, Martinelli, 2020, Donello et al., 2023, Dektor, 2024, Jacot, 2022, Chen et al., 2018, Ierley et al., 2019). Across these settings, rank is not a single invariant: it may denote the dimension of a tangent function space, the rank of an observability or controllability distribution, the matrix or TT-rank required at time $t$, the bottleneck dimension of a nonlinear map, or the cross-sectional position of a trajectory within a population.

1. Scope and taxonomy

A useful way to organize the subject is to separate the object being ranked from the axis along which the rank is examined.

Setting	Rank object	Cross-time axis
General nonlinear models	$\operatorname{rank}_{f_\theta}(f^*)$	training data size $n$
Time-varying nonlinear systems	$\dim \Omega(t,x)$, $\dim \Delta(t,x)$	explicit time $t$
Time-dependent basis ROMs	$\operatorname{rank}(V(t))$ or $r^k$	continuous or discrete time
Low-rank TT integration	TT-ranks $\boldsymbol r$	time $t$
Deep homogeneous networks	$\operatorname{rank}_{f_\theta}(f^*)$0, $\operatorname{rank}_{f_\theta}(f^*)$1, $\operatorname{rank}_{f_\theta}(f^*)$2	depth $\operatorname{rank}_{f_\theta}(f^*)$3
Functional and replicated temporal data	$\operatorname{rank}_{f_\theta}(f^*)$4, $\operatorname{rank}_{f_\theta}(f^*)$5, $\operatorname{rank}_{f_\theta}(f^*)$6	time $\operatorname{rank}_{f_\theta}(f^*)$7 or epoch index

This taxonomy reflects explicit constructions in the cited works (Zhang et al., 2022, Martinelli, 2020, Donello et al., 2023, Dektor, 2024, Jacot, 2022, Chen et al., 2018, Ierley et al., 2019). A recurring pattern is that rank becomes function-specific, architecture-specific, or state-dependent rather than a global count of raw parameters. This suggests that “cross-time nonlinear rank” is best understood as a family of localized rank notions for nonlinear systems whose effective dimension evolves or becomes identifiable only along an external axis.

2. Function-specific model rank and sample-size phase transitions

For a differentiable nonlinear model

$\operatorname{rank}_{f_\theta}(f^*)$8

the model rank at a parameter point $\operatorname{rank}_{f_\theta}(f^*)$9 is the dimension of the tangent function space,

$n$0

If a realizable target function $n$1 has multiple parameterizations, with target stratifold

$n$2

then the model rank of the function is

$n$3

This rank is the paper’s “effective size of parameters” for a specific function, not the raw parameter count $n$4. The resulting rank stratification decomposes parameter space and function space into subsets with different effective ranks (Zhang et al., 2022).

The central cross-time claim is a phase transition in target recovery as training sample size $n$5 increases. For data $n$6, the empirical tangent matrix is

$n$7

with empirical model rank

$n$8

If $n$9 is a global interpolating minimizer, then linear stability is equivalent to

$\dim \Omega(t,x)$0

Under analyticity and generic data, the paper proves that if

$\dim \Omega(t,x)$1

the target is not linearly stable, whereas if

$\dim \Omega(t,x)$2

the target is linearly stable almost everywhere. The interval

$\dim \Omega(t,x)$3

is the quasi-determined regime: globally overparameterized, yet locally unique in the tangent hyperplane. A common misconception addressed directly by this theory is that successful recovery is controlled by $\dim \Omega(t,x)$4; the paper instead argues that the relevant threshold is the target’s nonlinear model rank (Zhang et al., 2022).

Concrete instances make the point explicit. For matrix factorization $\dim \Omega(t,x)$5 with $\dim \Omega(t,x)$6, parameter size is $\dim \Omega(t,x)$7, but a target matrix of matrix rank $\dim \Omega(t,x)$8 has model rank

$\dim \Omega(t,x)$9

For a two-layer fully connected tanh network

$\dim \Delta(t,x)$0

functions realized with exactly $\dim \Delta(t,x)$1 nonredundant neurons have rank $\dim \Delta(t,x)$2, independent of available width $\dim \Delta(t,x)$3. For a two-layer $\dim \Delta(t,x)$4-kernel CNN, the same function has rank $\dim \Delta(t,x)$5 in the CNN, but much larger rank in a CNN without weight sharing or a fully connected network; on MNIST-like sizes with $\dim \Delta(t,x)$6 and $\dim \Delta(t,x)$7 kernels, the approximate ranks are $\dim \Delta(t,x)$8, $\dim \Delta(t,x)$9, and $t$0, respectively. In this framework, convolution and weight sharing reduce nonlinear model rank and therefore reduce the sample-size threshold at which linear stability emerges (Zhang et al., 2022).

3. Time-augmented rank conditions in nonlinear control

For time-varying nonlinear systems

$t$1

the relevant rank notion is built on the augmented time-state manifold. Introducing

$t$2

turns the system into an autonomous system on $t$3. This yields the mixed observability operator

$t$4

and the mixed controllability bracket

$t$5

These operators are the key cross-time modification: they couple explicit time variation with the usual Lie-derivative and Lie-bracket structure (Martinelli, 2020).

The observable codistribution is generated by

$t$6

starting from $t$7. The controllability distribution is generated by

$t$8

starting from $t$9. The extended observability rank condition states that if the converged $\operatorname{rank}(V(t))$0 is non-singular and $\operatorname{rank}(V(t))$1, then the system is weakly locally observable at $\operatorname{rank}(V(t))$2; the extended controllability rank condition states the analogous result for $\operatorname{rank}(V(t))$3 and weak local controllability. These conditions reduce to the classical time-varying linear tests and to the standard nonlinear time-invariant rank conditions in the appropriate limits (Martinelli, 2020).

The lunar module example shows the practical meaning of the construction. The system has a 14-dimensional state, explicit time dependence through fuel-induced changes in mass and inertia, and outputs from a monocular camera. Applying the extended observability rank condition gives $\operatorname{rank}(V(t))$4, so the full state is not weakly locally observable; the unobservable direction corresponds to infinitesimal rotation about the vertical axis, so the system is weakly locally observable up to the yaw angle. Applying the extended controllability rank condition yields a 12-dimensional controllability distribution whose orthogonal codistribution is spanned by $\operatorname{rank}(V(t))$5 and the quaternion-norm constraint, implying weak local controllability under constant main thrust. A common confusion is that time augmentation makes time itself controllable; the paper explicitly shows that controllable directions remain in the state subspace (Martinelli, 2020).

4. Rank-adaptive evolution on low-rank manifolds

In nonlinear stochastic PDEs, discretization in space and random parameters produces a matrix differential equation

$\operatorname{rank}(V(t))$6

A time-dependent basis reduced-order model seeks

$\operatorname{rank}(V(t))$7

The paper emphasizes that the effective rank is not constant in time: smooth transients, shocks, bifurcations, or changing stochastic structures can increase or decrease the number of significant singular values. Its TDB-CUR method replaces continuous-time projected evolution by the time-discrete minimization

$\operatorname{rank}(V(t))$8

whose exact solution is the rank-$\operatorname{rank}(V(t))$9 truncated SVD, and then approximates that solution by oblique projection using sparse row and column samples. The method is equivalent to a CUR decomposition, uses DEIM or QDEIM for sampling, avoids $r^k$0, is robust in the presence of small singular values, and updates rank using the proxy

$r^k$1

with user-prescribed bounds $r^k$2 and $r^k$3 (Donello et al., 2023).

The same paper gives a concrete cross-time picture in the stochastic Burgers equation. With rank adaptivity starting from $r^k$4 and tolerance $r^k$5, the rank increases in time to about $r^k$6 and then stays nearly constant. The growth is associated with more complex shock and stochastic structures. The DEIM/QDEIM sampling points cluster near the stochastic boundary and shock locations, and the adaptive procedure reduces error relative to fixed-rank evolution. A common misconception in this area is that low-rank time integration requires a fixed rank; the paper instead treats $r^k$7 as a time-dependent variable that should be adjusted on-the-fly (Donello et al., 2023).

A closely related development appears for nonlinear differential equations on low-rank tensor train manifolds. There, the solution $r^k$8 is evolved with interpolatory oblique projectors onto the tangent space, built by TT-cross-DEIM. Two schemes are proposed: a TT-cross integrator, which evolves the solution at selected interpolation indices and reconstructs it by cross interpolation, and an interpolatory projector-splitting integrator, which generalizes orthogonal projector-splitting to interpolatory projectors. The key technical point is that only entrywise evaluations of $r^k$9 at selected indices are required; no low-rank structure of the vector field is assumed. Rank adaptivity is driven by

$\boldsymbol r$0

with rank increase implemented by enlarging interpolation sets and rank decrease by TT-SVD truncation (Dektor, 2024).

5. Depth-dependent nonlinear rank in deep networks

For fully connected networks with homogeneous activation, the fundamental quantity is the representation cost

$\boldsymbol r$1

Its infinite-depth rescaling,

$\boldsymbol r$2

is shown to behave like a rank functional for finite piecewise linear functions. Two nonlinear rank notions are used to bound it: $\boldsymbol r$3 and

$\boldsymbol r$4

the minimal inner dimension of a factorization $\boldsymbol r$5 by finite piecewise linear functions. The central sandwich theorem is

$\boldsymbol r$6

For affine maps, $\boldsymbol r$7 equals the ordinary matrix rank. The paper also establishes submultiplicativity and subadditivity properties analogous to classical rank inequalities (Jacot, 2022).

Depth then plays a time-like role. For too large depths, the global minimum is approximately rank 1; the paper derives an approximate rank-1 regime in both Jacobian and bottleneck senses, including a hidden representation matrix $\boldsymbol r$8 at some layer with

$\boldsymbol r$9

At the same time, there is a range of depths which grows with the number of datapoints where the true rank of the data is recovered. This produces a three-regime picture: relatively high effective rank at small depth, a rank-recovery window at intermediate depth, and collapse toward rank 1 as $t$0 for fixed finite data (Jacot, 2022).

The depth-dependent rank has geometric consequences. When a classifier has bottleneck rank 1, its class boundaries are the pullback of a one-dimensional classification; generic tripoints disappear. For autoencoders, if the data lie on a $t$1-dimensional piecewise linear manifold and the learned autoencoder has optimal nonlinear rank $t$2, then locally it acts as an affine projection onto the data manifold, which the paper describes as natural denoising. A common misunderstanding is to identify this rank with width or parameter count; the paper instead ties it to the minimal bottleneck dimension of the realized function (Jacot, 2022).

6. Ordinal rank dynamics in functional and noisy time series

In functional data analysis, rank is defined observationally rather than parametrically. For a stochastic process $t$3 with cross-sectional CDF

$t$4

the cross-sectional rank trajectory of subject $t$5 is

$t$6

Its derivative decomposes as

$t$7

where

$t$8

Here $t$9 is the population component and $\operatorname{rank}_{f_\theta}(f^*)$00 the individual component. The framework adds summary measures such as integrated rank $\operatorname{rank}_{f_\theta}(f^*)$01, rank volatility $\operatorname{rank}_{f_\theta}(f^*)$02, net rank change $\operatorname{rank}_{f_\theta}(f^*)$03, rank movement $\operatorname{rank}_{f_\theta}(f^*)$04, time-varying rank instability $\operatorname{rank}_{f_\theta}(f^*)$05, and overall stability

$\operatorname{rank}_{f_\theta}(f^*)$06

The paper establishes joint asymptotic normality for suitable estimates of $\operatorname{rank}_{f_\theta}(f^*)$07 and $\operatorname{rank}_{f_\theta}(f^*)$08, making rank dynamics inferentially tractable (Chen et al., 2018).

A different observational construction appears in the rank-order transform for noisy data. Replicated time series are arranged into a rank-order data matrix, compressed into a rank–epoch matrix $\operatorname{rank}_{f_\theta}(f^*)$09, and then transformed to a stable form $\operatorname{rank}_{f_\theta}(f^*)$10 by comparing quadrant occupancies. The mean

$\operatorname{rank}_{f_\theta}(f^*)$11

is zero for stationary iid noise and varies monotonically with a linear trend, so one can estimate a slope by subtracting a trial trend until $\operatorname{rank}_{f_\theta}(f^*)$12. A group symmetry orthogonal decomposition of $\operatorname{rank}_{f_\theta}(f^*)$13, followed by PCA, yields a noise etalon for white noise; the resulting fingerprints distinguish the Ornstein–Uhlenbeck process and chaos generated by the logistic map from white noise. Because the method is based solely on rank information, it is nonparametric, objective, parsimonious, insensitive to outliers, effective in heavy-tailed noise, and not limited to linear regression. The same framework also gives a simple approximation for extracting a generally nonlinear signal,

$\operatorname{rank}_{f_\theta}(f^*)$14

from the rank transform (Ierley et al., 2019).

These two observational lines use “rank” in a different sense from model rank, controllability rank, or manifold rank. This suggests a broader interpretation of cross-time nonlinear rank: not only the intrinsic dimension of a nonlinear representation, but also the nonlinear temporal evolution of relative order itself.