---
title: Brain-Inspired Modular PINNs
url: https://www.emergentmind.com/topics/brain-inspired-modular-pinns-bimt
type: topic
---

# Brain-Inspired Modular PINNs

Brain-Inspired Modular PINNs (BIMT) are Physics-Informed Neural Network architectures explicitly designed to mimic the organization principles observed in biological brains, notably sparsity, locality, and modularity. By combining standard PINN loss functions with additional regularization and discrete optimization strategies, BIMT enables neural networks to self-organize into interpretable, energy-efficient modules while maintaining accuracy for scientific machine learning tasks such as PDE-solving. These principles are realized through Brain-Inspired Modular Training, which incorporates both connection cost penalties and locality-aware strategies, yielding highly modular and compact architectures. BIMT frameworks have demonstrated concrete benefits over traditional fully-connected approaches, including drastic reductions in parameter count, energy consumption, and enhanced mechanistic interpretability [2305.08746, 2310.07711, 2401.15661, 2511.06081].

## 1. Mathematical Formalism and Training Objective

The BIMT methodology augments the standard PINN objective with two essential regularization terms: global sparsity via $L_1$ penalty and anatomical locality via a connection-length-weighted $L_1$ cost. For a PDE in the abstract form $\mathcal{D}[u](\mathbf{x}) = f(\mathbf{x})$, the loss function for brain-inspired PINNs is:

\[
\mathcal{L}(\theta) =
\mathcal{L}_{\rm PDE} +
\mathcal{L}_{\rm BC} +
\lambda\sum_{\ell,i,j}\bigl|W_{ij}^{(\ell)}\bigr| +
A\sum_{\ell,i,j}d_{ij}^{(\ell)}\,\bigl|W_{ij}^{(\ell)}\bigr|
\]

where $\mathcal{L}_{\rm PDE}$ and $\mathcal{L}_{\rm BC}$ respectively encode the PDE residuals and boundary/initial condition violations, $\lambda$ controls sparsity, $A$ governs locality pressure, and $d_{ij}^{(\ell)}$ is the geometric distance between neuron $i$ and $j$ in layer $\ell$ [2401.15661, 2305.08746]. For networks solving compositional cognitive tasks or mechanistic regression, the same regularization principles apply, demonstrating that functional and anatomical modularity can co-emerge in both RNN-type and PINN-type architectures [2310.07711].

## 2. Enforcement of Locality, Sparsity, and Modular Connectivity

Explicit control over wiring length and sparsity is achieved by penalizing both absolute weight magnitudes and connection distances, measured in a fixed neuron embedding space (typically 2D or 3D grid). The connection cost term $C$ and augmented loss read:

\[
C = \sum_{i=1}^L \sum_{j=1}^{n_{i-1}}\sum_{k=1}^{n_i} d_{i,jk}\,\bigl|w_{i,jk}\bigr| + \sum_{i=1}^L\sum_{k=1}^{n_i} y_*\,\bigl|b_{i,k}\bigr|
\]
\[
L_{\rm total} = L_{\rm task} + \lambda\,C
\]

During training, occasional neuron-swapping steps are performed, exchanging the geometric positions of neuron pairs to minimize total connection cost without altering input-output functionality [2305.08746, 2401.15661]. Pruning post-training zeros out connections below a threshold, further revealing modular structure.

## 3. NeuroPINN: Modular Implementation with Variable Spiking Neurons

NeuroPINN instantiates the BIMT approach in a modular block structure:

- **Input-encoding module:** spatial-temporal coordinates, fed through spike-time-steps.
- **Spiking encoder:** layers of Dense→Variable Spiking Neuron (VSN), where VSN dynamics are governed by discrete membrane-potential updates, graded spiking, and hard resets:
  \[
  \mathcal M_{\bar t} = \beta_\ell\,\mathcal M_{\bar t-1} + z_{\bar t}
  \]
  Spiking occurs upon threshold crossing, with graded output activation, enabling sparse, event-driven communication.
- **Latent core & decoder:** further layers (spiking or continuous) and a final decoder to output the target field $\mathcal U(\mathbf x, t; \mathbf w)$.
- **Physics-residual & loss module:** stochastic projection (SP) approximates derivatives needed for PINN loss without backpropagation through spikes, while surrogate backpropagation ensures tractable gradient flow [2511.06081].

VSNs fire at empirically measured rates ($r=\{58\%, 52\%, 42\%, 29\%\}$), leading directly to reduced synaptic energy costs compared to conventional layers. Deployment on event-driven neuromorphic hardware enables up to 70% measured synaptic energy savings [2511.06081].

## 4. Emergence of Modular Building Blocks and Spectral Bias

BIMT training reliably discovers bare-minimum sub-networks that mimic canonical architectural primitives. For example:

- **Convolution-like modules:** restrict connectivity to local stencils, enforcing spatial locality.
- **Attention-like modules:** realize sparse, input-dependent modules with dot-product and gating mechanisms.
- **Voters and ring embeddings:** emerge in algorithmic tasks (modular addition, permutation group classification), manifesting as parallel, hierarchical, or tree-like structures [2305.08746].

The phenomenon of spectral bias is evident: networks targeting higher-frequency components in solution spaces require proportionally more hidden units. Experimental results on 1D Poisson problems show:

| Source Frequency $n$ | Minimal Hidden Units Required |
|----------------------|------------------------------|
| 1                    |       3                      |
| 2                    |       5                      |
| 4                    |      21                      |

This shows that BIMT-architectures naturally expand modular capacity as dictated by the underlying problem’s spectral content [2401.15661].

## 5. Metrics for Functional and Anatomical Modularity

Quantitative assessment involves:

- **Functional clustering:** e.g., $k$-means on normalized task-variance vectors per neuron.
- **Anatomical modularity:** fraction of isolated neurons, mean spatial cluster size, and comparison to random spatial assignments.

Lower isolation and compact cluster span quantitatively indicate increased anatomical modularity. BIMT always achieves Pareto-optimal trade-off in accuracy versus #active neurons and total wiring cost compared to vanilla $L_1$ regularization [2310.07711].

## 6. Practical Procedures and Experimental Outcomes

BIMT–PINN assembly follows this workflow:

1. **Initialization:** compact MLP, neurons mapped to spatial coordinates.
2. **Training phases:** progressive increase of sparsity and locality penalties.
3. **Swapping:** periodic neuron exchanges for cost minimization.
4. **Pruning:** zeroing of negligible weights for ultimate sparsity.

Architectural recommendations include minimal modules for low-frequency PDEs and increasingly wider, multi-module designs for high-frequency settings. BIMT–PINN maintains or slightly degrades accuracy (MSE differences $10^{-4}$ to $10^{-2}$ across tasks), with drastic reductions in parameter count and connectivity [2401.15661, 2305.08746].

| Problem                           | Standard PINN Error | BIMT-PINN Error | Modules Discovered      |
|------------------------------------|---------------------|----------------|------------------------|
| Logistic ODE                      | $\sim10^{-4}$       | $\sim10^{-4}$  | 1 hidden unit          |
| 1D Poisson ($\sin(t)$ source)      | $0.0094$            | $0.023$        | 3-unit submodules      |
| Modular Addition                   | 100% accuracy       | 100%           | 3 parallel voters      |
| MNIST classification               | 98.5%               | 98.0%          | Sparse “stroke detectors”|

## 7. Limitations, Scalability, and Future Directions

The most prominent limitations of current BIMT implementations are scale: evidence is restricted to relatively small nets (hundreds to low thousands of neurons); effects in large-scale architectures (e.g., transformers, LLMs) remain unproven [2305.08746]. Swapping operations can be computationally expensive for wide layers. LayerNorm and batch-norm are typically omitted, which may inhibit applicability to modern architectures.

Potential extensions include trainable embedding geometries, continuous relaxation of swapping (e.g., via Sinkhorn nets), modularity regularization for CNNs, selective application in transformer modules, and alternative embedding manifolds (spherical, hyperbolic).

## References

- “NeuroPINNs: Neuroscience Inspired Physics Informed Neural Networks” [2511.06081]
- “Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks” [2310.07711]
- “Brain-Inspired Physics-Informed Neural Networks: Bare-Minimum Neural Architectures for PDE Solvers” [2401.15661]
- “Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability” [2305.08746]

Source: https://www.emergentmind.com/topics/brain-inspired-modular-pinns-bimt