A likelihood-based framework for simultaneously learning both noise and growth dynamics using biologically-informed neural networks

Abstract: In recent years, neural ordinary differential equation frameworks such as Biologically-Informed Neural Networks (BINNs) have shown promise for learning mechanistic laws from sparse data. However, most existing approaches implicitly assume homoscedastic Gaussian noise, and therefore do not account for potentially meaningful structure in biological variability. Here, we present an extension to the existing BINNs framework that includes a learnable noise model, allowing discovery of the noise model directly from data. Using population growth as an example, we demonstrate that the framework accurately recovers the underlying noise structure and improves predictions of the underlying growth laws compared to existing approaches. As such, this work establishes a general likelihood-based framework for jointly learning dynamics and heteroscedastic noise within mechanistic neural network approaches.

• The paper introduces NLL–BINNs, which replace mean squared error with a learnable negative log-likelihood to infer growth dynamics and power-law noise, distinguishing additive, intermediate, and multiplicative variability.
• Synthetic tests show accurate recovery of trajectories, growth laws, and noise parameters, naturally calibrated 68% and 95% predictive intervals, and lower mechanistic error than RMSE-trained BINNs.
• The coral reef case study infers density-limited regrowth and strongly heteroscedastic noise from single trajectories, while highlighting challenges in identifiability, censoring, and mechanistic validation.

Motivation and contribution

Biologically-Informed Neural Networks (BINNs) embed partially known mechanistic structure—conservation laws, rate equations, density-dependent interactions—as soft constraints in a shared loss function, allowing unknown functional relationships to be learned from sparse biological data. A persistent limitation of this family of methods, and of neural ODE approaches more broadly, is the treatment of observation noise: the near-universal use of mean squared error losses implicitly imposes homoscedastic Gaussian noise, discarding potentially informative structure in biological variability. Crossley and Baker address this by introducing the NLL–BINN framework (2606.13475), which replaces the MSE data loss with a negative log-likelihood under an explicit, learnable noise model, so that both the governing dynamics and the noise structure are inferred simultaneously from data.

The central conceptual move is to treat noise as a mechanistic quantity rather than a nuisance. Variability in biological systems often scales with population size; by parameterising that scaling and learning it from data, the framework links statistical variability directly to underlying mechanisms.

Framework

The authors consider ODE models of the form $\mathrm{d}u/\mathrm{d}t = u\, g(u;\boldsymbol{p})$, with observations modelled as $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$, where $\epsilon \sim \mathcal{N}(0,1)$. The noise magnitude takes a power-law form:

$\sigma(u) = \sigma_0 |u|^{\alpha},$

which unifies additive ($\alpha=0$), multiplicative ($\alpha=1$), and intermediate heteroscedastic regimes within a single parametrisation. Both $\sigma_0$ and $\alpha$ are learned.

Training minimises a penalised likelihood: a data term given by the Gaussian NLL evaluated at the observed values $u_i^{\mathrm{o}}$ (a stable, fully observable approximation rather than evaluating variance at the latent state), plus an ODE residual loss on randomly sampled collocation points across the full time domain, plus a biological loss enforcing non-negativity of the predicted trajectory. The total loss is thus interpretable as maximum likelihood estimation subject to soft mechanistic constraints—a useful bridge between mechanistic neural differential equations and classical inverse-problem theory. The formulation is not restricted to Gaussian likelihoods; any distribution with tractable NLL could be substituted.

Implementation details are conventional: two MLPs (trajectory network $u_\theta$ and growth law network $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$0) with three hidden layers of 64 tanh units, Adam optimisation, mini-batching, validation-based early selection, and ensembles of ten seeds per replicate. Code is publicly available.

Recovery of dynamics and noise structure

Synthetic evaluation uses three sigmoidal growth models—the logistic, Gompertz, and Richards' models—chosen deliberately because they produce qualitatively similar trajectories despite distinct crowding functions, making them a challenging identification problem. The functional forms are never supplied to the BINN. Three noise regimes are tested: additive ($$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$1, $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$2), intermediate ($$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$3, $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$4), and multiplicative ($$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$5, $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$6).

Across all three models, the NLL–BINN recovers the smooth trajectory, the crowding function $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$7, and the noise profile $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$8 simultaneously. The learned parameters $$u_i^{\mathrm{o}} = u(t_i) + \epsilon\,\sigma(u(t_i))$$9 closely match ground truth in all regimes, with limited ensemble variability, demonstrating correct discrimination between additive, multiplicative, and intermediate noise structures.

Two sources of reduced identifiability are identified honestly. First, for densities below the initial condition $\epsilon \sim \mathcal{N}(0,1)$0, no observations exist, so learning there is driven only by the ODE and biological losses, admitting many solutions. Second, for the Richards' dataset with the largest noise magnitude, clipping at zero pushes observations above the carrying capacity $\epsilon \sim \mathcal{N}(0,1)$1 into regions unsupported by the true dynamics, again degrading identifiability. The authors also note a small bias in recovered $\epsilon \sim \mathcal{N}(0,1)$2 under additive noise, which supplementary results attribute to censoring introduced by clipping negative densities rather than to the framework itself; when negatives are retained, the bias largely disappears.

Uncertainty calibration

A notable result is that predictive intervals are well calibrated without any explicit coverage constraint during training: empirical coverage closely matches the nominal 68%/95% levels across all three noise regimes. This calibration emerges naturally from the likelihood-based data loss, supporting the claim that the probabilistic formulation yields reliable uncertainty quantification as a by-product of correct specification rather than post-hoc adjustment.

Comparison against RMSE-trained BINNs

To isolate the effect of the loss function, NLL–BINNs are compared to standard RMSE-trained BINNs with identical data splits and initialisations. Performance is measured by mechanistic error: the RMSE between the true noise-free dynamics and forward simulations using the learned growth law, rather than data misfit. This distinction matters because RMSE-trained models can fit noisy observations well while recovering biased dynamics.

The NLL–BINN consistently achieves lower median mechanistic error and reduced variability across replicates, with improvements pronounced for the logistic and Gompertz models but more modest for Richards'. The authors attribute the latter to practical non-identifiability arising from the extra shape parameter $\epsilon \sim \mathcal{N}(0,1)$3, which permits distinct parameter combinations to yield similar trajectories. The improvement pattern is consistent with the interpretation of NLL training as correctly specified likelihood weighting of observations, versus the implicit homoscedastic assumption of RMSE.

Application to coral reef data

The framework is applied to hard coral cover measurements at two reef sites near Lady Musgrave Island, Australia, following disturbance. These single-trajectory datasets lack replicates, so variability cannot be estimated independently—a setting where joint inference of dynamics and noise is essential. At both sites, the framework recovers sigmoidal regrowth, decreasing crowding functions consistent with density-limited growth (with site-to-site differences suggesting differing mechanisms or conditions), and increasing noise profiles indicative of heteroscedasticity. Notably, the inferred exponents exceed linear scaling ($\epsilon \sim \mathcal{N}(0,1)$4 and $\epsilon \sim \mathcal{N}(0,1)$5), i.e., stronger-than-multiplicative noise, extracted without replicated experiments. The authors caution that single-trajectory data limit unique identification of growth mechanisms, so the inferred dynamics represent one plausible explanation among several.

Limitations and open questions

Several limitations are acknowledged. The power-law noise parametrisation cannot capture more complex structures, such as the one-sided noise observed at small densities; more flexible noise representations via dedicated networks are possible but untested here. Identifiability trade-offs between learned dynamics and noise models are mitigated empirically through ODE constraints, regularisation, and ensembling, but a full characterisation of identifiability—particularly its dependence on data sparsity, noise magnitude, and parametrisation—remains open. The coral reef application demonstrates feasibility but not validated ground truth, and the super-linear exponents inferred there are unexplained mechanistically. Finally, the framework trades full posterior inference for computational efficiency; Bayesian extensions accounting for parameter uncertainty in both dynamics and noise remain future work, as does integration with universal differential equation settings.

Conclusion

This paper extends the BINN framework with a learnable, density-dependent noise model trained via negative log-likelihood, enabling simultaneous recovery of trajectories, governing laws, and noise structure. On synthetic sigmoidal growth problems it recovers noise parameters accurately, produces naturally calibrated uncertainty, and reduces mechanistic error relative to RMSE-trained baselines; on real coral reef data it extracts heteroscedastic noise structure from unreplicated single trajectories. The work repositions observation noise as an inferable mechanistic component within mechanistic neural differential equation learning, while leaving open the systematic study of identifiability under sparsity and noise magnitude.