Neural Basis Models (NBMs)

• Neural Basis Models are neural network architectures that use shared, learned basis functions to represent complex behaviors with fewer parameters.
• They leverage basis decomposition to enable clear visualization of feature effects and improve computational efficiency across tasks.
• Applications include interpretable machine learning, probabilistic forecasting, physics-informed solvers, neural network compression, and computational neuroscience.

Neural Basis Models (NBMs) are a class of neural network architectures and modeling strategies that employ a shared set of basis functions—typically parameterized by neural networks or composed with analytical structures—to induce compactness, interpretability, and architectural scalability across a wide range of domains. Leveraging basis decomposition, NBMs achieve statistical and computational efficiency by distilling complex network behaviors into structured combinations of global, reusable functional components. This paradigm subsumes several contemporary directions, including interpretable machine learning models, distributional and quantile regression frameworks, neural network compression, and physics-informed neural solvers.

1. Foundations and Formal Definition

NBMs generalize the idea of basis function expansion, widely used in function approximation, by parameterizing a small set of nonlinear bases (e.g., $h_k(\cdot)$, $k=1,\dots,B$) that are shared across inputs, features, parameters, or model layers. Each task-specific entity—such as a feature-wise shape function, distribution parameter, or per-layer weight tensor—is expressed as a linear (or, occasionally, affine) combination of these global bases with learned, context-specific coefficients.

A prototypical univariate NBM for a regression or classification task with $D$ features takes the form

$f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$

with $h_k$ parameterized via a shared neural network and $\alpha_{j,k}$ the feature-specific linear weights. The final prediction is additive: $\hat{y} = \beta_0 + \sum_{j=1}^D f_j(x_j).$ This additive structure facilitates global, faithful plotting of learned feature effects and scales parameter count as $O(N_{\text{basis}} + D \cdot B)$, a substantial improvement over per-feature networks or ensembles (Radenovic et al., 2022).

NBMs can be specialized for stepwise distributional regression (NBMLSS) (Brusaferri et al., 2024), quantile multi-horizon forecasting (QNBM) (Brusaferri et al., 17 Sep 2025), analytical basis embedding in PDE solvers (Alkhalifah et al., 2023), or even global network compression via tensor decompositions (Obukhov et al., 2020). In computational neuroscience, NBMs may refer to closed-loop models of embodied brain–body–environment systems amenable to information-theoretic analysis (Candadai, 2021).

2. Mathematical Formulations Across NBM Variants

NBMs manifest domain-appropriate formulations retaining the core shared-basis principle:

• Additive Predictive Models: Each feature's shape function is a linear combination of basis functions as above. Extensions include NB²M (bivariate basis expansions for modeling pairwise interactions) and multi-class versions with class-specific linear combinations (Radenovic et al., 2022).
• Distributional Regression (NBMLSS): For $n_p$ distribution parameters indexed by $p$ (e.g., location, scale, shape), and a multi-horizon setup with steps $k=1,\dots,B$0,

$k=1,\dots,B$1

where $k=1,\dots,B$2 is as above, $k=1,\dots,B$3 are horizon/parameter/feature weights, $k=1,\dots,B$4 enforces parameter constraints (e.g., softplus for positivity), and all bases are shared via a unified neural network (Brusaferri et al., 2024).

• Quantile Regression (QNBM): For quantile level $k=1,\dots,B$5 and horizon $k=1,\dots,B$6, prediction is

$k=1,\dots,B$7

subject to low-rank weight factorization and dropout for regularization and efficiency (Brusaferri et al., 17 Sep 2025).

• Analytical/Physics-Informed NBMs: The neural architecture outputs amplitudes that mix fixed analytical bases (e.g., parameterized Gabor wavelets) to enforce solution properties of physical systems:

$k=1,\dots,B$8

where $k=1,\dots,B$9 are neural-network-derived amplitudes, and $D$0 are Gabor functions dynamically centered by an auxiliary network (Alkhalifah et al., 2023).

• Neural Network Compression (T-Basis): All core tensors in multi-layer decompositions are constrained to reside in a global basis subspace:

$D$1

ensuring extreme compression ratios and parameter-sharing across the network (Obukhov et al., 2020).

3. Interpretability, Scalability, and Regularization

NBMs unify scalable learning and interpretable modeling:

• Interpretability: Additive NBMs permit direct plotting of $D$2, exposing monotonicities, saturations, and global feature effects. In distributional or quantile NBMs, the influence of each feature on each prediction dimension (step, quantile, parameter) can be visualized as heat-maps or curves, enabling actionable insight for practitioners (Radenovic et al., 2022, Brusaferri et al., 2024, Brusaferri et al., 17 Sep 2025).
• Basis Sharing and Parameter Efficiency: Global bases drastically reduce parameterization compared to per-feature or per-output MLP architectures, scaling well to large $D$3 or multi-output problems. For example, a standard Neural Additive Model (NAM) with $D$4 features and MLPs of size $D$5 has $D$6 parameters, whereas an NBM with basis dimension $D$7 and network size $D$8 has $D$9, yielding 10–50$f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$0 reduction at $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$1 (Radenovic et al., 2022).
• Regularization and Factorization: Weight decay, dropout, and specifically basis-dropout are employed to decorrelate basis functions and prevent overfitting. In high-dimensional NBMs, both basis and projection weight matrices are often approximated via low-rank factorizations (e.g., LoRA-style), controlling memory, compute, and effective variance (Brusaferri et al., 17 Sep 2025).
• Stability: Empirically, ensemble runs with NBMs yield more stable learned shape functions than NAMs, with an order-of-magnitude lower per-feature standard deviation (Radenovic et al., 2022).
• Sparse and Structured Feature Support: NBMs efficiently handle sparse feature spaces; only active features require forward computation through the shared basis network.

4. Applications and Empirical Performance

Interpretable Machine Learning

NBMs achieve state-of-the-art accuracy among generalized additive models for tabular regression, binary/multiclass classification, and structured sparse tasks (e.g., text, image attributes), while scaling efficiently to high $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$2 (Radenovic et al., 2022). NB²M achieves near–black-box accuracy on multiclass datasets.

Probabilistic Forecasting

NBMLSS and QNBM architectures underpin multi-horizon, probabilistic modeling for electricity markets, where interpretability and nonparametric flexibility are critical (Brusaferri et al., 2024, Brusaferri et al., 17 Sep 2025). NBMLSS replaces O($f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$3) networks of classical GAMLSS with a single shared basis, matching or exceeding the CRPS/MAE of deep distributional frameworks, while providing transparent shape functions and weight visualization. QNBM achieves comparable performance to parametric and quantile DNNs for day-ahead electricity price forecasting, with precise interval coverage and robust tail control.

Physics-Informed Neural Solvers

NBMs equipped with analytical bases (e.g., Gabor functions) enable efficient high-frequency solutions for PDEs such as the Helmholtz equation. Analytical Gabor NBMs drastically mitigate spectral bias, requiring fewer layers, epochs, and lower parameter counts for wavefield prediction, as evidenced by order-of-magnitude reductions in test error compared to vanilla PINNs (Alkhalifah et al., 2023).

Neural Network Compression

T-Basis enables global tensor-ring compression for deep architectures. By enforcing a shared basis of $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$4 cores, networks achieve $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$5–$f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$6 compression ratios on ResNet and segmentation models, with minimal accuracy degradation. Rank and basis size tuning are essential for optimal expressiveness (Obukhov et al., 2020).

Computational Neuroscience and Information Flow

NBM, as used in computational neuroscience, models the full closed-loop dynamics of agent–environment systems, allowing direct computation of entropy, mutual information, transfer entropy, and partial information decomposition to analyze the emergence and transformation of neural codes and their behavioral consequences (Candadai, 2021).

5. Limitations and Theoretical Considerations

Key limitations and open questions include:

• Concurvity and Identifiability: NBMs can suffer from concurvity (non-identifiability among correlated features), leading to ambiguous attributions. Remedies may involve penalization or regularized feature selection (Brusaferri et al., 17 Sep 2025).
• Interaction Modeling: Standard NBMs do not natively model higher-order feature interactions. Extensions employ pairwise or higher-arity basis decompositions (NB²M, NB³M), but at the expense of interpretability and parameter growth (Radenovic et al., 2022).
• Parameter Explosion in Dense Regimes: When $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$7 or for $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$8 interaction terms, even $f_j(x_j) = \sum_{k=1}^B \alpha_{j,k}\, h_k(x_j),$9 parameterization may become prohibitive, necessitating feature or basis selection (Radenovic et al., 2022).
• Domain-Specific Tailoring: Selection of basis dimension ($h_k$0, $h_k$1), factorization rank $h_k$2, and activation nonlinearity are critical and typically chosen via grid or Bayesian search, monitored by validation loss and interpretability metrics.
• Model Abstraction: In computational neuroscience NBMs, tractability comes at the cost of biological realism. The simplifying assumptions of CTRNNs or spiking neuron models do not capture the full complexity of neural tissue (Candadai, 2021).

A theoretical result demonstrates that if each learned feature function lies in an RKHS with rapidly decaying eigenvalues, $h_k$3 shared bases suffice to approximate the additive model within arbitrarily small error (Radenovic et al., 2022).

6. Future Directions and Extensions

Current NBMs serve as platforms for multiple avenues of active research:

• Basis-Enriched Modeling: Incorporation of richer analytical or domain-informed basis families (e.g., wavelets, polynomials, Green’s functions) for specialized structure or physics as in PINN applications (Alkhalifah et al., 2023).
• Multivariate and Adaptive Bases: Exploration of dynamic basis selection, adaptive basis dimensions per feature or task, and multivariate basis extensions to address dimensionality and context-sensitivity.
• Robust and Adaptive Uncertainty Quantification: Integrating conformal recalibration or adaptive distributional regression for finite-sample calibrated predictions and resilience under distribution shift (Brusaferri et al., 17 Sep 2025).
• Multi-agent and Social Interaction Modeling: Extending NBM frameworks to multi-agent systems for studying social information flow, synergy, and redundancy in distributed settings (Candadai, 2021).
• Hybrid Compression-Pruning Strategies: Merging T-Basis with quantization, pruning, or structured sparsity for further compression gains (Obukhov et al., 2020).
• Neural Basis Models Beyond Additivity: Generalizations toward non-additive combinations or non-linear mixing of bases, potentially approaching full black-box model flexibility while retaining partial interpretability.

7. Summary Table of Major NBM Variants

NBM Variant	Core Decomposition	Domain/Application
Additive NBM	Univariate bases + linear mix	Interpretable ML, tabular
NBMLSS	Shared bases, parameter-wise mixing	Distributional time series
QNBM	Shared bases, horizon–quantile mixing	Probabilistic forecasting
Analytical NBM	Neural amplitudes × analytical bases	Physics-informed/PINN
T-Basis Compression	Tensor-ring global subspace	Network compression
Embodied NBM (Neuro)	Dynamical neural–body–env. system	Computational neuroscience

In conclusion, Neural Basis Models constitute a versatile framework uniting interpretability, compactness, and performance, with principled applicability ranging from structured statistical modeling to neural compression and mechanistic neuroscience (Radenovic et al., 2022, Brusaferri et al., 2024, Brusaferri et al., 17 Sep 2025, Alkhalifah et al., 2023, Candadai, 2021, Obukhov et al., 2020).