Brain-Inspired Modular PINNs

• Brain-Inspired Modular PINNs (BIMT) are physics-informed neural network frameworks that mimic biological brain organization through enforced sparsity, locality, and modular connectivity.
• They integrate additional regularization terms, including L1 penalties and connection-length costs, to self-organize into modular, energy-efficient architectures while maintaining PDE accuracy.
• Experimental outcomes reveal substantial energy savings, lower parameter counts, and enhanced mechanistic interpretability compared to traditional fully-connected PINN models.

Brain-Inspired Modular PINNs (BIMT) are Physics-Informed Neural Network architectures explicitly designed to mimic the organization principles observed in biological brains, notably sparsity, locality, and modularity. By combining standard PINN loss functions with additional regularization and discrete optimization strategies, BIMT enables neural networks to self-organize into interpretable, energy-efficient modules while maintaining accuracy for scientific machine learning tasks such as PDE-solving. These principles are realized through Brain-Inspired Modular Training, which incorporates both connection cost penalties and locality-aware strategies, yielding highly modular and compact architectures. BIMT frameworks have demonstrated concrete benefits over traditional fully-connected approaches, including drastic reductions in parameter count, energy consumption, and enhanced mechanistic interpretability (Liu et al., 2023, Liu et al., 2023, Markidis, 2024, Garg et al., 8 Nov 2025).

1. Mathematical Formalism and Training Objective

The BIMT methodology augments the standard PINN objective with two essential regularization terms: global sparsity via $L_1$ penalty and anatomical locality via a connection-length-weighted $L_1$ cost. For a PDE in the abstract form $\mathcal{D}[u](\mathbf{x}) = f(\mathbf{x})$, the loss function for brain-inspired PINNs is:

$$\mathcal{L}(\theta) = \mathcal{L}_{\rm PDE} + \mathcal{L}_{\rm BC} + \lambda\sum_{\ell,i,j}\bigl|W_{ij}^{(\ell)}\bigr| + A\sum_{\ell,i,j}d_{ij}^{(\ell)}\,\bigl|W_{ij}^{(\ell)}\bigr|$$

where $\mathcal{L}_{\rm PDE}$ and $\mathcal{L}_{\rm BC}$ respectively encode the PDE residuals and boundary/initial condition violations, $\lambda$ controls sparsity, $A$ governs locality pressure, and $d_{ij}^{(\ell)}$ is the geometric distance between neuron $i$ and $j$ in layer $\ell$ (Markidis, 2024, Liu et al., 2023). For networks solving compositional cognitive tasks or mechanistic regression, the same regularization principles apply, demonstrating that functional and anatomical modularity can co-emerge in both RNN-type and PINN-type architectures (Liu et al., 2023).

2. Enforcement of Locality, Sparsity, and Modular Connectivity

Explicit control over wiring length and sparsity is achieved by penalizing both absolute weight magnitudes and connection distances, measured in a fixed neuron embedding space (typically 2D or 3D grid). The connection cost term $C$ and augmented loss read:

$$C = \sum_{i=1}^L \sum_{j=1}^{n_{i-1}}\sum_{k=1}^{n_i} d_{i,jk}\,\bigl|w_{i,jk}\bigr| + \sum_{i=1}^L\sum_{k=1}^{n_i} y_*\,\bigl|b_{i,k}\bigr|$$

$L_{\rm total} = L_{\rm task} + \lambda\,C$

During training, occasional neuron-swapping steps are performed, exchanging the geometric positions of neuron pairs to minimize total connection cost without altering input-output functionality (Liu et al., 2023, Markidis, 2024). Pruning post-training zeros out connections below a threshold, further revealing modular structure.

3. NeuroPINN: Modular Implementation with Variable Spiking Neurons

NeuroPINN instantiates the BIMT approach in a modular block structure:

• Input-encoding module: spatial-temporal coordinates, fed through spike-time-steps.
• Spiking encoder: layers of Dense→Variable Spiking Neuron (VSN), where VSN dynamics are governed by discrete membrane-potential updates, graded spiking, and hard resets:

$$\mathcal M_{\bar t} = \beta_\ell\,\mathcal M_{\bar t-1} + z_{\bar t}$$

Spiking occurs upon threshold crossing, with graded output activation, enabling sparse, event-driven communication.

• Latent core & decoder: further layers (spiking or continuous) and a final decoder to output the target field $\mathcal U(\mathbf x, t; \mathbf w)$.
• Physics-residual & loss module: stochastic projection (SP) approximates derivatives needed for PINN loss without backpropagation through spikes, while surrogate backpropagation ensures tractable gradient flow (Garg et al., 8 Nov 2025).

VSNs fire at empirically measured rates ($r=\{58\%, 52\%, 42\%, 29\%\}$), leading directly to reduced synaptic energy costs compared to conventional layers. Deployment on event-driven neuromorphic hardware enables up to 70% measured synaptic energy savings (Garg et al., 8 Nov 2025).

4. Emergence of Modular Building Blocks and Spectral Bias

BIMT training reliably discovers bare-minimum sub-networks that mimic canonical architectural primitives. For example:

• Convolution-like modules: restrict connectivity to local stencils, enforcing spatial locality.
• Attention-like modules: realize sparse, input-dependent modules with dot-product and gating mechanisms.
• Voters and ring embeddings: emerge in algorithmic tasks (modular addition, permutation group classification), manifesting as parallel, hierarchical, or tree-like structures (Liu et al., 2023).

The phenomenon of spectral bias is evident: networks targeting higher-frequency components in solution spaces require proportionally more hidden units. Experimental results on 1D Poisson problems show:

Source Frequency $n$	Minimal Hidden Units Required
1	3
2	5
4	21

This shows that BIMT-architectures naturally expand modular capacity as dictated by the underlying problem’s spectral content (Markidis, 2024).

5. Metrics for Functional and Anatomical Modularity

Quantitative assessment involves:

• Functional clustering: e.g., $k$-means on normalized task-variance vectors per neuron.
• Anatomical modularity: fraction of isolated neurons, mean spatial cluster size, and comparison to random spatial assignments.

Lower isolation and compact cluster span quantitatively indicate increased anatomical modularity. BIMT always achieves Pareto-optimal trade-off in accuracy versus #active neurons and total wiring cost compared to vanilla $L_1$ regularization (Liu et al., 2023).

6. Practical Procedures and Experimental Outcomes

BIMT–PINN assembly follows this workflow:

1. Initialization: compact MLP, neurons mapped to spatial coordinates.
2. Training phases: progressive increase of sparsity and locality penalties.
3. Swapping: periodic neuron exchanges for cost minimization.
4. Pruning: zeroing of negligible weights for ultimate sparsity.

Architectural recommendations include minimal modules for low-frequency PDEs and increasingly wider, multi-module designs for high-frequency settings. BIMT–PINN maintains or slightly degrades accuracy (MSE differences $10^{-4}$ to $10^{-2}$ across tasks), with drastic reductions in parameter count and connectivity (Markidis, 2024, Liu et al., 2023).

Problem	Standard PINN Error	BIMT-PINN Error	Modules Discovered
Logistic ODE	$\sim10^{-4}$	$\sim10^{-4}$	1 hidden unit
1D Poisson ($\sin(t)$ source)	$0.0094$	$0.023$	3-unit submodules
Modular Addition	100% accuracy	100%	3 parallel voters
MNIST classification	98.5%	98.0%	Sparse “stroke detectors”

7. Limitations, Scalability, and Future Directions

The most prominent limitations of current BIMT implementations are scale: evidence is restricted to relatively small nets (hundreds to low thousands of neurons); effects in large-scale architectures (e.g., transformers, LLMs) remain unproven (Liu et al., 2023). Swapping operations can be computationally expensive for wide layers. LayerNorm and batch-norm are typically omitted, which may inhibit applicability to modern architectures.

Potential extensions include trainable embedding geometries, continuous relaxation of swapping (e.g., via Sinkhorn nets), modularity regularization for CNNs, selective application in transformer modules, and alternative embedding manifolds (spherical, hyperbolic).

• “NeuroPINNs: Neuroscience Inspired Physics Informed Neural Networks” (Garg et al., 8 Nov 2025)
• “Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks” (Liu et al., 2023)
• “Brain-Inspired Physics-Informed Neural Networks: Bare-Minimum Neural Architectures for PDE Solvers” (Markidis, 2024)
• “Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability” (Liu et al., 2023)