Subspace Basis Fusion

• Subspace basis fusion is a method that combines multiple subspace representations into a unified framework used in signal processing, machine learning, and neural model adaptation.
• It leverages fusion frames, Grassmannian geometry, and operator theory to ensure optimal reconstruction and minimal coherence across different data sources.
• The approach underpins practical applications such as adaptive clustering, low-rank model updates, and robust multimodal data fusion in modern neural systems.

Subspace basis fusion is a mathematical and algorithmic paradigm that constructs a unified representation by combining the bases of multiple subspaces—often originating from distinct data sources, modalities, or model components—into a jointly structured or optimal subspace system. Across functional analysis, machine learning, signal processing, and neural model adaptation, subspace basis fusion unifies disparate local subspace bases into a global structure that preserves reconstruction, clustering, interpretability, or downstream task performance. The methodologies span classical fusion frames, atomic subspaces for operators, optimized clustering via Grassmannian distances, combinatorial designs for optimal subspace packing, and modern neural and data fusion frameworks.

1. Core Concepts and Mathematical Foundations

Subspace basis fusion begins with the classical theory of fusion frames. For a separable Hilbert space $H$, a family $\{(W_i, v_i)\}_{i\in I}$ with closed subspaces $W_i\subset H$ and positive weights $v_i$ is a fusion frame if there exist $0 < A \leq B < \infty$ satisfying

$$A\|f\|^2 \;\leq\; \sum_{i\in I} v_i^2 \|P_{W_i} f\|^2 \;\leq\; B\|f\|^2,\quad\forall f\in H,$$

where $P_{W_i}$ is the orthogonal projection onto $W_i$ (Bhandari et al., 2017). The fusion frame operator $S = \sum_i v_i^2 P_{W_i}$ is positive and invertible on $H$, and when $A = B$ the frame is tight.

Atomic subspaces generalize this by considering a bounded linear operator $K\in \mathcal{L}(H)$. The collection $\{(W_i, v_i)\}$ is atomic with respect to $K$ if (i) $\sum_{i} v_i f_i$ converges in $H$ for $\sum_i \|f_i\|^2 <\infty$ and (ii) every $f\in H$ admits $Kf = \sum_{i} v_i f_i$ for suitable $f_i \in W_i$ with quadratic control (Bhandari et al., 2017). When $K=I$, this reduces to a standard fusion frame.

Modern perspectives on subspace basis fusion broaden the framework from orthogonal projections to general or operator-induced subspace analysis and synthesis mappings, $U$ and $U^*$, and more generally to g-fusion frames employing local analysis operators $\Theta_k : H \to \mathcal{M}_k$ (Jahedi et al., 2023).

2. Operator-Theoretic and Structural Characterizations

Subspace basis fusion possesses rich operator-theoretic characterizations. The synthesis operator $T_W : \bigoplus_{i} W_i \to H$, $T_W(\{f_i\}) = \sum_{i} v_i f_i$, and analysis operator $T_W^* : H \to \bigoplus_{i} W_i$, $T_W^* f = (v_i P_{W_i} f)$, enable fusion frames to be studied via operator inequalities: $A\, K K^* \leq T_W T_W^* \leq B\, I_H,$ with the K-fusion frame property guaranteeing surjectivity of $T_W$ on $\operatorname{Range}(K)$ and the existence of a bounded left-inverse $L$ with $K = T_W L$ (Bhandari et al., 2017). The fusion frame operator $S_W = T_W T_W^*$ is positive and invertible on the subspace of interest. In g-fusion frames, additional structure is provided by possibly non-self-adjoint or surjective $\Theta_k$ operators (Jahedi et al., 2023).

Properties of direct sums and intersections are central. If each $\{(W_i^{(j)}, v_i)\}_{i\in I}$ is a $K_j$-fusion frame for $H_j$, then $\{(W_i^{(1)}\oplus\cdots\oplus W_i^{(m)}, v_i)\}$ is a $(K_1\oplus\cdots\oplus K_m)$-fusion frame for $H_1\oplus\cdots\oplus H_m$. Intersection stability, under commuting projections, allows construction of new fusion frames by intersecting each $W_i$ with a closed subspace $V$ (Bhandari et al., 2017).

3. Fusion via Grassmannian Geometry and Optimal Packing

A unifying metric for subspace basis fusion is the Grassmannian or chordal distance. For $K$-dimensional subspaces $W_i$, $W_j$ with projectors $P_i$, $P_j$, the squared chordal distance is

$$d^2_c(W_i, W_j) = K - \operatorname{Tr}(P_i P_j) = \frac{1}{2}\|P_i-P_j\|_F^2$$

(King, 2010). This metric underlies optimal subspace packing: Grassmannian fusion frames are those that maximize the minimal chordal distance among all $N$ $K$-dimensional subspaces in $\mathbb{F}^M$. Tight Grassmannian fusion frames, particularly those achieving the Welch or orthoplex bounds, minimize the subspace coherence parameter $\mu_s = \max_{i\neq j} \operatorname{Tr}(P_i P_j)$, which is essential for robustness to erasures and noise.

Explicit constructions, such as those using Hadamard matrices, show that block-wise orthogonalizations of columns partitioned from the Walsh-Hadamard matrix produce equi-isoclinic, tight Grassmannian fusion frames. These frames exhibit optimal pairwise separation, robustness, and minimal coherence (King, 2010). Recent work extends these concepts to mixed-rank packings on the Grassmannian using traceless embeddings and combinatorial block designs (Casazza et al., 2019, Bodmann et al., 2016).

4. Algorithmic and Learning-Based Fusion Mechanisms

Subspace basis fusion appears centrally in modern machine learning. Fusion Subspace Clustering (FSC) algorithms assign an adaptive subspace to every data point and use a convex fusion penalty on the Grassmannian (Frobenius norm of projector differences) to iteratively merge bases serving similar data. The optimization problem is

$$\sum_i \|x_i - P_i x_i\|^2 + \frac{\lambda}{2} \sum_{i,j} \|P_i - P_j\|_F^2,$$

where increasing $\lambda$ fuses subspaces (Pimentel-Alarcón et al., 2018, Mahmood et al., 2022). This approach scales to missing-data settings by restricting projections to observed coordinates, allowing information-theoretic optimal clustering rates and smoothly controlling the number of active subspaces.

In multimodal and high-dimensional data fusion, joint representations are obtained by enforcing group-sparsity across self-representation matrices (RoGSuRe), extracting fused bases from principal components of grouped modalities and enhancing clustering accuracy over naive feature concatenation (Ghanem et al., 2020).

Operator representations in fusion frames are further extended in applications such as Schatten class and Hilbert-Schmidt operators, where fusion-frame tensors decompose the functional space of operators and yield blockwise matrix representations compatible with pseudo-inversion and decomposition theory (Balazs et al., 2020).

5. Fusion in Neural Model Adaptation and Parameter-Efficient Learning

Neural adaptation and model merging apply subspace basis fusion at the parameter level. In LoRA and its extensions, trainable low-rank matrices $(A,B)$ form a subspace basis for updates on frozen backbone weights. SRLoRA dynamically fuses directions of low importance (rank-1 basis vectors) into the backbone and reinitializes new basis directions along unused SVD principal directions, maintaining a fixed subspace adaptation budget while enlarging the effective fusion span over time and improving downstream performance (Yang et al., 18 May 2025).

In LLM safety realignment (SOMF), subspace basis fusion disentangles task deltas, learns a safety subspace via a coordinate-wise probabilistic mask, and fuses the safe and task-specific components back onto the base model using a tunable fusion operator. This matrix-level fusion preserves safety across multiple tasks while maintaining instruction-following or code capability (Yi et al., 2024).

In training-time neuron alignment, permutation subspaces (as defined by fixed masking over coordinates) are used to restrict SGD evolution and break post-hoc symmetry in neuron indexing, yielding subspaces with aligned basis elements suitable for barrier-free fusion by linear combination or averaging. These methods yield higher accuracy in model soups and federated learning contexts (Li et al., 2024).

6. Advanced Topics: Multimodal, High-Order, and Knowledge-Driven Fusion

Recent multimodal approaches utilize knowledge-driven subspace fusion with biologically interpretable decompositions (e.g., tumour vs. microenvironment), deformable cross-modal attention, and gradient coordination across paired subspaces. These strategies yield embeddings where the basis elements of each subspace are explicitly fused via cross-attention driven by modal-specific teachers, with learned consistency losses and dynamic, confidence-weighted gradient updates to maintain interpretability and performance (Zhang et al., 2024).

For joint high-dimensional features (e.g., CNN and LOMO in person re-identification), tensor fusion schemes assemble disparate feature matrices into a unified third-order tensor, and multilinear subspace learning (e.g., TXQDA) seeks mode-wise orthonormal subspace bases via generalized eigenproblems, yielding a fused low-dimensional basis that integrates multiple views and enhances cross-view discrimination (Chouchane et al., 9 May 2025).

Subspace basis fusion also underpins self-learning diffusion models for HSI-MSI image fusion, where decoupled spatial and spectral bases are iteratively fused by light-weight diffusion networks, with residual-guided correction ensuring globally coherent reconstruction (Zhu et al., 17 May 2025).

7. Connections, Path-Connectedness, and Gradient-Based Tightening

The global structure of the space of fusion frames underlies the ability to move between local and fused bases. The fusion frame homotopy theorem states that, when tight fusion frames exist with given subspace dimensions, the space of all such frames is path-connected: all frames can be reached from one another via continuous deformation within the space (Needham et al., 2022). This enables the use of geometric optimization—via the fusion frame potential functional, which attains its global minimum at tight fusion frames—and justifies deterministic gradient-descent flows on the product of Grassmannians to explicitly "fuse" arbitrary local subspace bases into a unified tight global structure, free of spurious minima (Needham et al., 2022).

---

References:

• (Bhandari et al., 2017) Atomic subspaces for operators
• (Pimentel-Alarcón et al., 2018) Fusion Subspace Clustering: Full and Incomplete Data
• (Mahmood et al., 2022) Fusion Subspace Clustering for Incomplete Data
• (King, 2010) Grassmannian Fusion Frames
• (Casazza et al., 2019) A notion of optimal packings of subspaces with mixed-rank and solutions
• (Bodmann et al., 2016) Maximal Orthoplectic Fusion Frames from Mutually Unbiased Bases and Block Designs
• (Jahedi et al., 2023) On $g-$Fusion Frames Representations via Linear Operators
• (Balazs et al., 2020) Representation of Operators Using Fusion Frames
• (Yang et al., 18 May 2025) SRLoRA: Subspace Recomposition in Low-Rank Adaptation via Importance-Based Fusion and Reinitialization
• (Yi et al., 2024) A safety realignment framework via subspace-oriented model fusion for LLMs
• (Li et al., 2024) Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion
• (Ghanem et al., 2020) Robust Group Subspace Recovery: A New Approach for Multi-Modality Data Fusion
• (Zhu et al., 17 May 2025) Self-Learning Hyperspectral and Multispectral Image Fusion via Adaptive Residual Guided Subspace Diffusion Model
• (Chouchane et al., 9 May 2025) Multilinear subspace learning for person re-identification based fusion of high order tensor features
• (Zhang et al., 2024) Knowledge-driven Subspace Fusion and Gradient Coordination for Multi-modal Learning
• (Needham et al., 2022) Fusion Frame Homotopy and Tightening Fusion Frames by Gradient Descent