Core Space Merging

Updated 10 December 2025

Core space merging is the process of identifying, aligning, and merging central subspaces from diverse models to preserve key informational structures.
It employs methods like SVD-based decomposition and joint basis alignment to fuse low-rank representations in neural networks and spatial clustering in astrophysical simulations.
This approach improves multitask accuracy by 1–8% in neural models and enhances efficiency in mapping galactic cores, while reducing computational complexity.

Core space merging is a methodology for combining or tracking the central, information-rich subspaces—termed "core spaces"—across various scientific domains, including neural network adaptation, astrophysical simulations of galaxy and molecular cloud evolution, and large-scale halo modeling in cosmology. The unifying element is the identification, alignment, and coalescence of "core" regions or subspaces that encapsulate the dominant structural, dynamical, or parametric content relevant to a system’s subsequent evolution or task performance. This framework has undergone recent, rapid formalization in large neural network merging (Panariello et al., 22 Sep 2025, Chaichana et al., 29 May 2025), cosmological substructure organization (Vitório et al., 2024), molecular cloud evolution (Nejad-Asghar, 2010), and galactic core formation (Rantala et al., 2018).

1. Definition and Scope of Core Space Merging

Core space merging is defined as the process of constructing, aligning, and unifying the central subspaces or compact representations—“cores”—associated with distinct models, physical bodies, or halo substructures in a way that preserves or enhances system-level properties (such as predictive accuracy, dynamical stability, or structural correspondence) under constraints on computation, information, or physical evolution. The underlying methodologies vary by discipline but share abstract components: core identification (typically via geometric, energetic, or algebraic criteria), projection into a joint or reference space, and a merging operator acting in this reduced or highly informative space.

In neural networks and machine learning, core space merging is operationalized as merging in the common alignment basis extracted (often via SVD) from a set of low-rank task adapters, yielding maximal efficiency and minimal information loss (Panariello et al., 22 Sep 2025, Chaichana et al., 29 May 2025). In astrophysics and cosmology, core merging may refer to processes whereby gravitationally bound cores in galaxies or molecular clouds form larger entities through physically motivated merger or scouring events (Nejad-Asghar, 2010, Rantala et al., 2018, Vitório et al., 2024).

2. Mathematical Foundations and Representational Formulation

In neural model merging, let $W_0 \in \mathbb{R}^{m \times n}$ be the base network weight, and $\Delta W^{(t)} = B^{(t)} A^{(t)}$ the LoRA-style low-rank update for task $t$ ( $A^{(t)} \in \mathbb{R}^{r \times n}$ , $B^{(t)} \in \mathbb{R}^{m \times r}$ , $r \ll \min(m,n)$ ). The procedure is as follows (Panariello et al., 22 Sep 2025):

SVD Decomposition: Compute factor SVDs, then define per-task core matrices $M^{(t)}$ in the reduced singular bases.
Reference Core Space Alignment: Stack the LoRA factors across $T$ tasks and perform SVD to obtain a common orthonormal basis $(U_B^{\mathrm{ref}}, V_A^{\mathrm{ref}})$ spanning the union of all tasks’ core directions.
Projection Operators: Map each $M^{(t)}$ into this shared core space using closed-form alignment coefficients $R_B^{(t)}, Q_A^{(t)}$ .
Merging Function: Perform merging (arithmetic, geometric, or more complex nonlinear operations) in the shared core space; then reconstruct the full merged weight by applying the inverse mapping.

The formal guarantee is that, provided $Tr \leq m, n$ (total number of core space directions), projection into and reconstruction from the shared core space is information-preserving (zero alignment error), as shown analytically in (Panariello et al., 22 Sep 2025). DRM further generalizes this to full-rank settings, where all fine-tuned weight deltas for $N$ tasks are horizontally stacked and their SVD computed to define a maximal "core space" for each layer (Chaichana et al., 29 May 2025).

In cosmological simulations, core space merging is realized via spatial and mass-based identification: Friends-of-Friends halo finders select tightly bound "core particles," which are then grouped by proximity and infall mass thresholds to construct one-to-one matches with galaxies or subhalos (Vitório et al., 2024).

3. Core Space Merging Algorithms and Pseudocode

Canonical Neural Core Space Merging Algorithm (Panariello et al., 22 Sep 2025):

A_stack = np.vstack([A_1, ..., A_T])      # (T*r x n)
B_stack = np.hstack([B_1, ..., B_T])      # (m x T*r)
U_B_ref, _, _ = np.linalg.svd(B_stack)    # (m x T*r)
_, _, V_A_ref = np.linalg.svd(A_stack)    # (n x T*r)

for t in range(T):
    M_t = ...                             # Core matrix for task t via SVD
    R_B_t = U_B_ref.T @ U_B_t
    Q_A_t = V_A_ref.T @ V_A_t
    M_t_aligned = R_B_t @ M_t @ Q_A_t.T

merged_core = merge_core_matrices([M_1_aligned, ..., M_T_aligned])
Delta_W_merged = U_B_ref @ merged_core @ V_A_ref.T
W_merged = W_0 + Delta_W_merged

DRM-H Core Space Merging (Chaichana et al., 29 May 2025):

Stack $\Delta W^{(t)}$ horizontally, SVD to get $U, \Sigma, V^T$ for the layer.
Subdivide $V^T$ into block rows per task and renormalize rows to unit norm.
Merge by (optionally pruned, sign-selected, or averaged) reweighting across tasks, then reconstruct the merged update as $U (\Sigma \tilde{V}^T)^{\mathrm{M}}$ .

In both approaches, merging is performed entirely in a high-density, information-aligned space rather than in the original weight basis, ensuring maximal efficiency and improved multitask accuracy.

Spatial core-group merging in astrophysics (Vitório et al., 2024) is implemented as a FoF search in core-particle coordinate space, followed by infall-mass-based pruning and catalog construction.

4. Empirical and Computational Characterization

Core space merging achieves a significant reduction of computational complexity relative to conventional full-parameter merging schemes. For T tasks, model dimension $n$ , and LoRA rank $r$ :

Strategy	Dominant Operation	Time Complexity ( $\mathcal{O}$ )
Full Space	Direct assembly/merging in $n\times n$	$\mathbf{n^2Tr}$ , $n^3$ , $n^3T$
KnOTS Space	SVD on $n \times nT$ stack	$n^3T^2$ , $n^3T^2 + n^2Tr$
Core Space	SVD on $(Tr)\times n$ and SVD alignment/merge	$\mathbf{n^2Tr} + T^3r^3$

For $Tr \ll n$ , core space merging provides at least an order-of-magnitude reduction in merge-time. Empirically, in Llama 3 8B merges, merge times drop from thousands to tens of seconds, with normalized multitask accuracy often 1–8% higher than best prior methods depending on architecture and task type (Panariello et al., 22 Sep 2025, Chaichana et al., 29 May 2025). Renormalization is identified as central to the stability and efficacy of the core space, with ablation studies showing accuracy drops of 4–9% if omitted (Chaichana et al., 29 May 2025).

5. Physical Core Space Merging in Astrophysics and Cosmology

Astrophysical core space merging mechanisms are unified by their focus on the dynamical coalescence of central structures, whether these are:

Binary SMBH-driven core scouring in giant ellipticals: Where merger-induced slingshot ejection of stars leads to a low-density, tangentially biased core, the size of which scales with SMBH mass, and with velocity anisotropy parameter $\beta$ becoming strongly negative (tangential) (Rantala et al., 2018).
Low-mass condensation (LMC) mergers in molecular clouds: Where gravitational, pressure-confined clumps merge on $\lesssim10^5$ yr timescales, forming protostellar clusters with altered mass spectra and energy evolution. Simple inelastic collision criteria and predictor–corrector N-body integration are used to simulate structural evolution and core collapse (Nejad-Asghar, 2010).
Halo core merging for galaxy–halo mapping: Core identification followed by spatial FoF linkage with optimized linking-length $b$ and minimum infall mass cut allows a faithful, efficient mapping between simulation cores and empirical galaxy counts and distributions at the $\sim10\%$ level (Vitório et al., 2024).

These applications demonstrate that core space merging, whether in configuration or energy/momentum space, drives both local and global structure formation, with system-level observables (core radii, mass spectra, structural stability) determined largely by merger and alignment processes at the core level.

6. Impact, Limitations, and Future Directions

Core space merging unifies approaches for parameter-efficient model fusion in machine learning, robust substructure mapping in cosmology, and physical structure formation via mergers in astrophysics. Its main benefits are:

Maximal preservation of task or structural information under low-rank or SVD-induced projections, given exactness theorems for aligned core spaces (Panariello et al., 22 Sep 2025);
Computational scalability to many tasks, large parameter spaces, or massive simulations (Panariello et al., 22 Sep 2025, Chaichana et al., 29 May 2025, Vitório et al., 2024);
Accurate empirical recovery of target observables (e.g., multitask accuracy, substructure count, radial profiles) compared to baselines (Panariello et al., 22 Sep 2025, Chaichana et al., 29 May 2025, Vitório et al., 2024).

Principal limitations arise when tasks or substructures are non-overlapping (orthogonal cores), when the shared core space becomes prohibitively large ( $Tr \to n$ ), or when SVD or FoF methods become numerically unstable or algorithmically less efficient (Chaichana et al., 29 May 2025, Vitório et al., 2024). In physical contexts, energy injection from inelastic merger events or the onset of collective instability (e.g., protostellar collapse or binary hardening) introduces new dynamical constraints not seen in parameter merging.

Further research is expected to refine core space alignment (e.g., via spectral regularization, automated pruning, adaptive merger criteria), extend the framework to mixed-architecture settings, and probe the limits of information retention and computational feasibility as both network and cosmological data volumes grow. The cross-disciplinary applicability of core space merging underscores its role as a foundational technique in modern multiscale modeling.