Nested Subspace Chains in Hierarchical Models

• Nested subspace chains are ordered collections of subspaces that structure hierarchical representations in fields such as coding theory, machine learning, and geometric combinatorics.
• They enable efficient optimization on flag manifolds and Riemannian descent methods, ensuring monotonic inclusion and improved consistency across multiple scales.
• Their applications span adaptive deep learning architectures, hierarchical locally recoverable codes, and infinite-dimensional function approximation, fostering flexible computational models.

A nested subspace chain (NSS) is a sequence of subspaces (or affine subspaces) within a vector space, metric space, or algebraic structure, totally ordered by inclusion. The NSS concept organizes representations, hierarchical decompositions, and computational hierarchies across diverse fields, from geometric combinatorics and representation learning to coding theory and infinite-dimensional functional approximation. In contemporary machine learning and information theory, nested subspace chains and their algorithmic analogs underpin models with hierarchical adaptation, multiresolution recovery, and consistent multiscale representation—allowing precise control over model capacity, computational cost, and error.

1. Definitions and Structural Properties

A nested subspace chain in an ambient space $X$ is a sequence

$$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$

where each $A_i$ is a subspace (or affine subspace). The general form, appearing in machine learning and combinatorics, admits several variants:

• Affine subspace chains (hierarchical codes): For $V = \mathbb{F}_q^m$ and $A_j = v + S_j$, with $S_1 \subset \cdots \subset S_h = V$ a flag of linear subspaces, the chain $A_1 \subset A_2 \subset \cdots \subset A_h$ is called a nested affine subspace chain (Haymaker et al., 2023).
• Chains of subspaces in lattice geometry: In a point–line geometry $G$, any totally ordered subset $C \subset \operatorname{Sub}(G)$ by inclusion forms a nested subspace chain; its length is $|C|-1$ (Pasini, 2019).
• Flag manifolds (machine learning): The flag manifold $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$0 consists of $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$1-tuples of subspaces $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$2 such that $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$3 (Szwagier et al., 9 Feb 2025).
• Nested subspace arrangements (representational learning): $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$4 structures the possible $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$5-level containment chains in metric or inner-product spaces (Hata et al., 2020).

These chains serve as a universal language for hierarchical structure, enabling both theoretical characterization (rank, capacity) and practical algorithmic constructions.

2. Hierarchical Representation and Optimization in Machine Learning

Nested subspace chains are foundational in algorithms enforcing hierarchy, consistency, and adaptivity in representations:

• Hierarchical Subspace Optimization: Traditional low-dimensional representation methods (e.g., PCA, CCA) optimize over the Grassmannian $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$6. This yields independent subspaces for different $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$7, which may not be nested. The flag trick lifts optimization to the flag manifold, ensuring $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$8, thereby achieving monotonic inclusion and consistency across scales (Szwagier et al., 9 Feb 2025).
• Riemannian Optimization: The flag manifold $$A_1 \subset A_2 \subset \cdots \subset A_n \subset X$$9 is a smooth manifold, enabling efficient Riemannian steepest-descent algorithms using blockwise gradients and QR/polar retraction, preserving nestedness at every step and converging rapidly, typically in tens of iterations.
• Applications:
  • Nested PCA identifies the unique global minimum corresponding to the leading eigenvectors of the covariance matrix for each specified dimension (Szwagier et al., 9 Feb 2025).
  • Nested CCA produces canonical subspaces with nested structure, allowing extraction of multilevel interpretability in multi-view data.

Empirical evidence shows that enforcing nestedness via flag optimization improves cross-rank consistency, avoids non-monotonic variance, and enhances downstream task performance vs. methods that train individual ranks independently.

3. Adaptive Deep Learning Architectures: Nested Subspace Networks

Nested subspace property has been leveraged to introduce architectural adaptability in deep neural networks:

• Nested Subspace Networks (NSNs): Each linear layer $A_i$0 is reparameterized as $A_i$1, with $A_i$2, $A_i$3, such that for every $A_i$4, $A_i$5 (Rauba et al., 22 Sep 2025). This produces a hierarchy $A_i$6 at the level of images, enforcing the nested subspace property

$A_i$7

• Joint Hierarchical Training: All ranks are optimized jointly with uncertainty-weighted cross-entropy losses, where learnable uncertainties $A_i$8 balance training across ranks. The uncertainty-weighted loss is critical; ablations confirm that omitting this component causes severe collapse at lower ranks.
• Fine-Grained Compute–Accuracy Tradeoffs: At inference, the rank parameter can be chosen dynamically to fit a target FLOPs budget, enabling smooth, continuous control over accuracy vs. efficiency.
• Surgical Adaptation of Pre-trained Models: NSNs can be applied post-hoc to arbitrary foundation models by SVD factorization and fine-tuning with only a few epochs.

Empirically, a single NSN can match the accuracy–FLOPs curve of many specialist models, enabling $A_i$9 reduction in inference FLOPs with only $V = \mathbb{F}_q^m$0 percentage points loss in accuracy on high-resource tasks. This paradigm enables instant test-time adaptability, post-hoc applicability, and a continuous tradeoff frontier (Rauba et al., 22 Sep 2025).

4. Theoretical and Geometric Foundations in Combinatorics and Geometry

Nested subspace chains clarify the relationship between generators, independence, and rank in combinatorial and geometric settings:

• Chains and Rank Equivalence: In combinatorial geometries where the subspace lattice $V = \mathbb{F}_q^m$1 satisfies the Exchange Property (EP), the generating rank $V = \mathbb{F}_q^m$2 matches the supremum of lengths of well-ordered subspace chains (Pasini, 2019). Arbitrary chains can be strictly longer in infinite-dimensional settings, making well-ordering essential for equivalence with algebraic notions of rank.
• Importance of Well-Ordered Chains: For projective and polar spaces, maximal well-ordered chains of singular subspaces yield the correct notion of rank, generalized to infinite settings.
• Critical Lemmas:
  • Any independent set of size $V = \mathbb{F}_q^m$3 produces a well-ordered chain of length $V = \mathbb{F}_q^m$4.
  • Conversely, any well-ordered chain of length $V = \mathbb{F}_q^m$5 produces an independent set of the same size.
• Applications in Polar Spaces: The polar rank can be defined via supremum of lengths of well-ordered chains of singular subspaces, enabling structural theorems unifying combinatorial and geometric viewpoints.

This theoretical apparatus underpins the structure of nested subspaces in algebraic geometry, combinatorial design, and incidence geometry.

5. Information Theory and Hierarchical Locally Recoverable Codes

Nested subspace chains are also instrumental in the construction and analysis of locally recoverable codes (LRCs) with hierarchical recovery:

• Hierarchical Recovery Structures: Given a sequence of affine subspace chains $V = \mathbb{F}_q^m$6 through a point $V = \mathbb{F}_q^m$7, middle codes $V = \mathbb{F}_q^m$8 are defined as the restriction of the global code to the $V = \mathbb{F}_q^m$9-flat. This induces locality at multiple levels: fine repair (low $A_j = v + S_j$0) for small erasures, coarse repair (high $A_j = v + S_j$1) for larger erasure patterns (Haymaker et al., 2023).
• Explicit Parameters: In Reed–Muller codes, for each level $A_j = v + S_j$2, code parameters $A_j = v + S_j$3 can be computed explicitly, with $A_j = v + S_j$4 and distance $A_j = v + S_j$5 scaling accordingly.
• Hierarchical Interpolation: Recovery is accomplished by interpolating univariate/bivariate/multivariate polynomials on the subspaces $A_j = v + S_j$6, leveraging the nested structure to escalate to higher-dimensional recovery as needed.
• Unification of Families: Fiber-product codes, Artin–Schreier codes, and Reed–Muller codes are unified as instances of the same nested subspace chain principle.
• Advantages: This approach yields uniformity, explicit repair capability at all levels, and flexible tuning of locality vs. minimum distance (Haymaker et al., 2023).

Nested subspace chains enable explicit, tractable design of codes with multilayered availability and recovery guarantees.

6. Representation Learning: Nested Subspace Arrangements and Embeddings

The nested subspace arrangement (NSS) framework generalizes modern relational embedding methods:

• General NSS Arrangement: For nodes $A_j = v + S_j$7 in a relational dataset $A_j = v + S_j$8, an NSS embedding assigns to each $A_j = v + S_j$9 a chain $S_1 \subset \cdots \subset S_h = V$0 in a metric space, with relations reconstructed by inclusion/membership among the $S_1 \subset \cdots \subset S_h = V$1 (Hata et al., 2020).
• Unifying Role: Well-known embedding models (Euclidean, Poincaré, inner-product, TransE, disk-embedding) emerge as degenerate cases under choices of $S_1 \subset \cdots \subset S_h = V$2, $S_1 \subset \cdots \subset S_h = V$3, and reconstruction rules.
• DANCAR Model: The Disk-Anchor Arrangement specializes NSS to $S_1 \subset \cdots \subset S_h = V$4 (a point and a disk), enabling precise, high-fidelity reconstruction of large-scale directed graphs (e.g., WordNet with F1=0.993 in $S_1 \subset \cdots \subset S_h = V$5). The approach captures both hierarchical reachability and community structure via containment geometry.
• Learning and Optimization: Loss functions combine hinge or ReLU losses for positive/negative pairs and anchor regularization, efficiently optimized with Adam and batchwise negative sampling.
• Visualization and Interpretability: Disk sizes and anchor positions encode node "influence" and reachability, producing interpretable 2D and higher-dimensional representations of complex graphs.

NSS arrangements thus provide a rigorous, generalizable, and efficient geometric language for embedding large and richly structured relational data.

7. Infinite-Dimensional Approximation via Nested Subspace Sampling

Intractability in infinite-variate $S_1 \subset \cdots \subset S_h = V$6-approximation is mitigated by algorithms that exploit chains of nested subspaces:

• Orthogonal Decomposition: In weighted RKHS of infinitely many variables, the space decomposes into orthogonal $S_1 \subset \cdots \subset S_h = V$7 indexed by finite $S_1 \subset \cdots \subset S_h = V$8. The nested chain

$S_1 \subset \cdots \subset S_h = V$9

is defined by $A_1 \subset A_2 \subset \cdots \subset A_h$0 (Harsha et al., 2023).

• NSS Cost Model: Sampling cost depends on the subspace level; algorithms select linear functionals living in $A_1 \subset A_2 \subset \cdots \subset A_h$1 with cost $A_1 \subset A_2 \subset \cdots \subset A_h$2(k)$$.</li> <li><strong>Optimal Approximation Algorithms:</strong> For ANOVA spaces (with orthogonal summands), blockwise SVD truncations yield globally optimal multilevel methods. For non-ANOVA spaces, similar algorithms remain minimax optimal within bounds dependent on the decay of weights.</li> <li><strong>Polynomial Convergence Rates:</strong> The minimal error$$A_1 \subset A_2 \subset \cdots \subset A_h$3 decays as $A_1 \subset A_2 \subset \cdots \subset A_h$4, where $A_1 \subset A_2 \subset \cdots \subset A_h$5, with $A_1 \subset A_2 \subset \cdots \subset A_h$6 the decay of univariate eigenvalues and $A_1 \subset A_2 \subset \cdots \subset A_h$7 the decay of product weights.
• Implications: Regular or moderate weight decay ensures tractability, with computational cost scaling only polynomially in the accuracy.

This machinery underlies adaptive function approximation frameworks in high- and infinite-dimensional settings.

---

Nested subspace chains serve as a foundational mathematical and algorithmic principle unifying hierarchical representation, adaptive model design, multilevel statistical learning, geometric combinatorics, and hierarchical error recovery across multiple fields (Rauba et al., 22 Sep 2025, Szwagier et al., 9 Feb 2025, Haymaker et al., 2023, Harsha et al., 2023, Hata et al., 2020, Pasini, 2019).