Reliable mechanistic operator recovery with biologically-informed neural networks: principles for architecture and optimisation design

Abstract: Many biological processes are governed by complex dynamical mechanisms that remain incompletely understood despite increasing volumes of experimental data. Biologically-informed neural networks (BINNs) seek to address this challenge by embedding mechanistic differential equations into neural network training, enabling interpretable constitutive operators to be recovered directly from sparse and noisy observations. However, reliable operator recovery depends sensitively on network architecture, optimisation strategy, and data informativeness. Here, we present a systematic empirical study of how these factors influence mechanistic inference using BINNs applied to canonical one-dimensional advection-diffusion-reaction partial differential equation models. Across a suite of benchmark problems, we investigate how network expressivity, learning rate, loss weighting, and batch size influence optimisation behaviour and operator recovery. We show that successful mechanistic inference depends on balancing competing objectives rather than maximising any single aspect of the model or optimisation. Moderately expressive architectures outperform overly complex networks, intermediate learning rates improve optimisation stability, balanced data and PDE losses are essential for accurate operator recovery, and intermediate batch sizes provide the best compromise between computational efficiency and reproducibility. We further identify practical diagnostics for recognising common failure modes, including over-fitting, unstable optimisation, and poor mechanistic recovery when the ground truth is unavailable. Together, these findings provide evidence-based guidelines for deploying BINNs as credible tools for biological model discovery.

• The paper presents a comprehensive empirical analysis showing that moderate network expressivity and balanced loss weighting are key to reliable operator recovery in BINNs.
• It demonstrates that appropriate hyperparameter tuning, including intermediate learning rates and batch sizes, significantly improves the fidelity of PDE-constrained inferences from sparse, noisy data.
• The study underscores that separating solution interpolation from numerical integration is crucial for ensuring mechanistic fidelity during model validation.

Principles and Practices for Reliable Mechanistic Operator Recovery with Biologically-Informed Neural Networks

Introduction

Biologically-informed neural networks (BINNs) integrate mechanistic PDE modeling with neural architecture, enabling the recovery of interpretable constitutive operators directly from sparse and noisy biological data. The paper "Reliable mechanistic operator recovery with biologically-informed neural networks: principles for architecture and optimisation design" (2607.07425) presents a systematic empirical exploration of architectural, optimisation, and data-driven factors determining the efficacy and robustness of operator recovery in BINNs. The authors analyze canonical one-dimensional advection-diffusion-reaction (ADR) PDEs, highlighting the delicate balance required across model expressivity, optimisation stability, loss weighting, and data informativeness for reliable inference.

Framework: ADR--BINN Formulation and Workflow

The ADR--BINN framework consists of multiple MLPs representing the solution and constitutive operators—diffusion ($D(u)$), advection ($V(u)$), and reaction ($G(u)$)—trained jointly through gradient descent with automatic differentiation. The loss function incorporates data-fitting, PDE residual, and boundary-condition terms, with stochastic collocation underpinning PDE and boundary evaluations, thereby regularizing training and mitigating over-fitting.

Figure 1: Architecture of ADR--BINN, showing the solution network, operator networks, and loss aggregation for joint optimisation and operator recovery.

A crucial workflow distinction is introduced: the learned solution $u_\theta(x,t)$ is directly interpolated by the network, while the forward-predicted solution $u_\mathrm{p}(x,t)$ is obtained by numerically integrating the recovered operators. This separation is essential; $u_\theta$ may closely fit noisy data, but only $u_\mathrm{p}$ reflects mechanistic fidelity.

Sensitivity to Noise and Data Informativeness

Binary experiments demonstrate that BINNs exhibit robust operator recovery under moderate Gaussian noise, remaining resilient provided network architecture is not overly expressive. Excess capacity yields increased susceptibility to noise amplification, particularly in PDE residuals governing operator estimation. With increasing noise variance or data sparsity, shallower architectures promote implicit regularisation but may limit expressivity for nonlinear or multimodal dynamics.

Figure 2: BINN performance under increasing Gaussian noise, illustrating robustness with moderate noise and vulnerability to over-fitting at higher variance.

Data informativeness is paramount; operator estimates are constrained only within the range of solution states covered by the training data. Extrapolation beyond these values is unreliable unless additional constraints are imposed. Increasing spatial/temporal sampling systematically reduces operator uncertainty, but optimal recovery is not guaranteed by data abundance alone.

Optimisation: Learning Rate, Loss Weighting, and Batch Size

Successful BINN optimisation hinges on careful hyperparameter tuning. Intermediate learning rates ($r \sim 10^{-4}$) consistently yield efficient exploration and stable convergence. Insufficient rates induce under-fitting, while excessive rates produce unstable or divergent loss trajectories.

Figure 3: Validation loss and operator recovery versus learning rate, showcasing optimal regimes for stable convergence.

Loss weighting between data-fitting and PDE residual terms is particularly critical. Underweighting the PDE term yields operators that interpolate data but lack mechanistic validity, leading to discrepancies between $u_\theta$ and $u_\mathrm{p}$. Excess weighting of the PDE residual sacrifices data fidelity, degrading solution and operator accuracy. Across all cases studied, retaining $V(u)$0 is essential; diagnostics based on $V(u)$1 versus $V(u)$2 discrepancies provide practical guidance.

Figure 4: Influence of PDE residual weighting ($V(u)$3), with balanced loss yielding optimal operator recovery and highlighting failure modes.

Batch size is another critical variable, trading off computational efficiency, optimisation stochasticity, and reproducibility. Intermediate sizes ($V(u)$4--$V(u)$5) maximize exploration and accuracy without excessive variability or resource demands.

Figure 5: Mini-batch size effects on optimisation speed, loss trajectories, and solution/operator accuracy.

Network Architecture: Minimal Expressivity Suffices

Operator networks need only moderate expressivity to recover nonlinear constitutive relationships. Once network depth and width suffice for the underlying complexity, additional parameters confer no benefit and may degrade robustness via over-fitting and increased variability across initialisations.

Figure 6: Constitutive operator recovery as a function of network architecture, highlighting the sufficiency of moderate complexity.

Diagnostics and Recommendations

Validation loss evolution, consistency between $V(u)$6 and $V(u)$7, and invariance across network initialisations constitute practical diagnostics for reliability, especially in experimental contexts lacking ground truth. The findings underscore the following recommendations for BINN deployment:

1. Choose moderately expressive architectures tailored to problem complexity, avoiding over-parameterisation.
2. Ensure loss functions balance data fitting and PDE constraint, with data-fitting weight at least comparable to the PDE residual.
3. Employ intermediate learning rates and batch sizes for maximal optimisation efficacy.
4. Use diagnostics based on $V(u)$8 vs. $V(u)$9 and loss trajectories to monitor operator validity.

Implications and Future Directions

The paper's systematic analysis presents a conceptual framework concretizing the joint influence of architecture, optimisation, and data on BINN performance, with direct ramifications for biological model discovery. It cautions against conflating interpolation accuracy with mechanistic validity and offers evidence-based guidelines for hyperparameter selection and operator interpretation.

Theoretical implications include the need for identifiability analysis, adaptive loss weighting, uncertainty quantification, and extensions to high-dimensional, heterogeneous, or multi-species systems. Practical extensions for AI-driven scientific discovery involve coupling BINNs with symbolic regression, Bayesian inference, and automated model selection, further bridging mechanistic theory and empirical data.

Conclusion

Reliable mechanistic inference via BINNs is governed not by maximal network complexity or brute-force optimisation but by judicious balance of expressivity, optimisation, and data informativeness. The principles delineated herein serve as a robust foundation for deploying BINNs in biological and broader scientific contexts, ensuring credible operator recovery and advancing the integration of mechanistic modeling with neural computation.