Spatially Embedded Neural Networks

• Spatially Embedded Neural Networks are architectures that incorporate physical positions and spatial constraints to shape network topology and connection strengths.
• They employ methods like wiring length minimization, spatial regularization, and adaptive masking to instill biologically inspired and efficient inductive biases.
• These models demonstrate practical benefits in domains such as neurobiology, geospatial prediction, and parameter-efficient deep learning with robust performance metrics.

Spatially embedded neural networks are neural architectures and learning models in which explicit spatial structures, constraints, or interactions fundamentally shape network topology, connection strengths, and functional dynamics. The spatial embedding can refer to the physical placement of nodes (e.g., neurons or regions), spatially dependent wiring rules, or the incorporation of spatial features into graph or coordinate-based neural computations. This embedding induces inductive biases—such as wiring cost minimization, delay-length regularization, or spatially local message-passing—that reflect both biological principles and practical considerations in real-world tasks ranging from neurobiology to geospatial modeling and parameter-efficient deep learning.

1. Foundational Principles and Motivation

Spatially embedded networks formalize the idea that the connectivity or operation of neural models is governed by the positions of their elements in physical or abstract space, rather than purely topological adjacency. Early work in biological network science (Stiso et al., 2018) established that neuron and region locations, and the associated wiring costs, fundamentally constrain brain connectomes. The core principles include:

• Wiring length minimization: Connectivity favors short range over long range to reduce metabolic, material, and conduction costs.
• Trade-offs with global efficiency: While purely local wiring produces lattice-like high-clustering graphs with long path lengths, efficient computation often requires long-range “shortcut” connections, yielding small-world architectures.
• Explicit spatial features: Nodes and edges may carry pointwise, regional, positional, or edge-profile features reflecting their spatial context, as in geospatial and environmental networks (Fan et al., 1 Feb 2025).

Spatial embedding thereby induces a spectrum of topological properties (modularity, clustering, rich clubs), influences communication delays, and introduces physically interpretable inductive biases.

2. Mathematical Formulations and Model Architectures

Spatial embedding can be implemented at various levels within neural models:

• Neuron embeddings: Each neuron $i$ acquires a position $p_i \in \mathbb{R}^d$; synaptic weights $w_{ij}$ are defined as some function of the Euclidean or manifold distance $d_{ij} = \|p_i - p_j\|_2$ (Erb et al., 16 Jun 2025, Mészáros et al., 3 Nov 2025).

$w_{ij} = \frac{1}{d_{ij}}$

• Delay-dependent computation: In spiking networks, the transmission delay $d_{ji}$ for a spike from $i$ to $j$ is set directly by spatial distance:

$d_{ji} = \|p_i - p_j\|_2$

• Spatial regularization in RNNs: Losses combined with wiring and communicability costs:

$$L_{\text{total}} = L_{\text{task}} + \lambda_1 \sum_{i,j} |W_{ij}| + \lambda_S \sum_{i,j} D_{ij}|W_{ij}| + \lambda_C \sum_{i,j} C_{ij}|W_{ij}|$$

where $D_{ij}$ is spatial wiring cost, $C_{ij}$ is communicability, and $W_{ij}$ the recurrent weights (Sheeran et al., 26 Sep 2024).

• Multimodal spatial graph networks: Comprehensive spatial feature fusion is realized in architectures such as GMu-SGCN, collecting node point features ($x_v$), regional features ($r_v$), position ($p_v$), and edge features ($e_{u,v}$) into message passing (Fan et al., 1 Feb 2025).

$$m^{l}_{v \to u} = \tilde{p}^{l}_{u|v} \circ x^{l-1}_v \circ \tilde{r}^{l}_v \circ \tilde{e}^{l}_{u,v}$$

• Spatially adaptive masking: In implicit neural representations and SIREN-style architectures, spatial masks $M^i(x)$ gate neurons' influence by coordinate, localizing high-frequency contribution to regions with local detail (Feng et al., 12 Mar 2025).

3. Computational Strategies and Optimization

Spatially embedded neural networks necessitate specialized computational approaches:

• Backpropagation through positions: Gradients are computed with respect to neuron coordinates $p_i$, requiring higher-order automatic differentiation for weights defined as $w_{ij} = d_{ij}^{-1}$ (Erb et al., 16 Jun 2025).
• Sparse neighbor graphs and mini-batching: Geospatial models utilize nearest-neighbor Gaussian Process (NNGP) graphs with scalable mini-batching and localized graph convolutions (Zhan et al., 2023).
• Spectral and entropic measures: Quantification of modularity (Newman-Girvan $Q$), Shannon entropy $H(W)$ of the learned weights, and spectral entropy $H_{\text{spec}}$ of the eigenvalues are crucial for characterizing the impact of spatial constraints (Sheeran et al., 26 Sep 2024).

Table: Core Regularization Terms in Spatially Embedded RNNs (Sheeran et al., 26 Sep 2024)

Term	Formula	Interpretation
Wiring cost	$D_{ij}|W_{ij}|$	Penalizes long connections
Communicability	$C_{ij}|W_{ij}|$, $C = \exp(S^{-1/2}WS^{-1/2})$	All-walk broadcast cost
Sparsity	$|W_{ij}|$	Favors compact connectivity

4. Empirical Outcomes and Applications

Spatially embedded architectures have demonstrated distinctive benefits and behaviors across domains:

• Biological realism and modularity: Networks trained with spatial cost spontaneously organize into modular, small-world topologies with functionally specialized clusters and low entropy (Mészáros et al., 3 Nov 2025, Sheeran et al., 26 Sep 2024). In spiking networks, distance-regularized delays induce small-worldness and stronger modularity (Ridge regression $R^2 \approx 0.8$ for input-to-position alignment) (Mészáros et al., 3 Nov 2025).
• Geospatial prediction and uncertainty: NN-GLS, a neural network embedded in Gaussian Process spatial models, achieves consistent estimation and valid prediction intervals even in irregular spatial designs, outperforming non-spatial methods on both simulated and real PM$_{2.5}$ data (Zhan et al., 2023).
• Efficiency and resilience: Parameter-efficient models with position-optimized neurons maintain competitive performance under extreme sparsity, outperforming traditional baselines of similar parameter count (e.g., at 95% sparsity, spatially embedded MLP exceeds baseline accuracy) (Erb et al., 16 Jun 2025).
• Detailed signal modeling: Spatially-adaptive SNNs (SASNet) robustly fit high-frequency image regions without overfitting smooth backgrounds, achieving substantial PSNR and SSIM gains (Feng et al., 12 Mar 2025).
• Environmental and infrastructure graphs: GMu-SGCN demonstrates multi-modal spatial feature fusion yields up to 37% higher edge prediction accuracy over baselines for power grid reconstruction (Fan et al., 1 Feb 2025).

5. Topological and Dynamical Implications

Spatial embedding systematically shapes network topology and dynamical behaviors:

• Topological constraints: Networks formed under spatial trade-offs exhibit hierarchical modularity, rich-club formation, and persistent homological cycles as predicted by persistent homology analysis (Stiso et al., 2018).
• Spectral signatures: Constrained spatial learning produces suppressed leading eigenvalues (damped dominant modes) and increased spectral entropy (heterogeneous dynamics), with eigenvalues collapsing to the real axis as matrix symmetry increases (Sheeran et al., 26 Sep 2024).
• Wiring principles in pruning: Magnitude-based pruning in spatial models typically removes long-distance connections first, reinforcing minimum wiring length motifs found in biological networks (Erb et al., 16 Jun 2025).
• Functional clustering: Functional specialization and region-based clustering emerge without explicit target losses, as neurons tuned to similar input features co-localize in embedding space (Mészáros et al., 3 Nov 2025).

6. Domain-Specific Implementations and Theoretical Guarantees

Spatial embedding is applied across multiple areas, each with tailored theoretical and computational analysis:

• Geostatistics: NN-GLS leverages generalized least squares with NNGP precision matrices, yielding consistent mean estimation and valid kriging prediction intervals for dependent spatial data (Zhan et al., 2023). Theoretical rates depend on neural network approximation error and working covariance conditioning.
• Biological neurosciences: Spatial generative models, elastodynamic simulations, and spatial null models are deployed to analyze and interpret connectome formation, epilepsy propagation patterns, and disease-associated wiring anomalies (Stiso et al., 2018).
• Environmental and infrastructure networks: Multimodal fusion of spatial features, as shown in GMu-SGCN, is essential for reconstructing connectivity in natural (river) and man-made (power grid) networks (Fan et al., 1 Feb 2025).

7. Open Problems and Research Directions

Key unresolved questions and avenues for future development include:

• Non-Euclidean embeddings: Extension to networks embedded in manifolds, with invariance guarantees under rotation, translation, and non-uniform geometric transformations (Zhang et al., 2023). The details remain elusive due to unavailable models.
• Integration of physical constraints: Explicit modeling of mechanical forces, laminar organization, and geometric packing in generative models to mirror developmental biology (Stiso et al., 2018).
• Extensions to new data modalities: Handling spatial gradients, non-dyadic features, and combining spatial embedding with control-theoretic objectives in both health and disease contexts.
• Scaling and interpretability: Balancing parameter efficiency, sparsity, and mechanistic interpretability in scalable architectures for edge computing and neuromorphic hardware (Erb et al., 16 Jun 2025, Mészáros et al., 3 Nov 2025).
• Unified spatial GNN frameworks: Multi-modal and distance-regularized message-passing schemes to capture complex spatial dependencies in heterogeneous graphs (Fan et al., 1 Feb 2025).

A plausible implication is that the systematic integration of spatial embedding across neural architectures can simultaneously enable more efficient, interpretable, and biologically plausible networks, yet further work is needed to generalize these results to non-Euclidean geometries and new types of spatial constraints.