Residual Neural Likelihood Estimation (RNLE)

• RNLE is a simulation-based inference methodology that models the conditional noise likelihood (residuals) directly for deterministic signal-plus-noise systems.
• It employs a Masked Autoregressive Flow architecture with multiple autoregressive transforms to accurately capture non-Gaussian noise characteristics and glitches.
• The method achieves simulation efficiency and unbiased parameter inference in challenging environments, outperforming traditional neural likelihood estimation approaches.

Residual Neural Likelihood Estimation (RNLE) is a simulation-based inference methodology that leverages neural density estimation to directly learn the likelihood distribution of noise (residuals) conditioned on model parameters, rather than the full data likelihood. Designed to exploit additive signal-plus-noise models common in scientific data, RNLE enables efficient and robust parameter inference—especially in contexts where noise is non-Gaussian or contaminated with realistic artifacts. It is particularly relevant for gravitational-wave astronomy but applies broadly to any domain with deterministically modeled signals embedded in complex, non-stationary noise (Emma et al., 20 Jan 2026).

1. Mathematical Foundations

Let $d$ denote the observed data (e.g., gravitational-wave detector strain time series), and let $\theta$ summarize the deterministic signal parameters. RNLE assumes an additive data generative model,

$d = m(\theta) + \epsilon,$

where $m(\theta)$ is the deterministic signal model and $\epsilon$ is the realization of the (potentially non-Gaussian) noise.

The full data likelihood $p(d|\theta)$, frequently intractable for realistic noise, is recast via the change of variable $\epsilon = d - m(\theta)$ as

$$p(d|\theta) = p(\epsilon = d - m(\theta) \mid \theta).$$

Thus, the task is shifted to learning the conditional likelihood of the residuals $p(\epsilon|\theta)$. RNLE approximates this distribution with a neural density estimator $q_\phi(\epsilon|\theta)$, where $\phi$ represents the network weights.

Model parameters $\phi$ are obtained by minimizing the KL divergence between the true and model noise distributions:

$$\phi^* = \underset{\phi}{\arg\min}\ \mathbb{E}_{\theta \sim \pi(\theta),\ \epsilon \sim p(\epsilon|\theta)}\left[ \mathrm{KL}\left(p(\epsilon|\theta) \| q_\phi(\epsilon|\theta)\right) \right],$$

which is equivalent to maximizing the expected log-likelihood of noise realizations under $q_\phi$. Empirically, for a training set $\{(\theta_i, \epsilon_i)\}$, the loss function is

$$L(\phi) = -\frac{1}{N} \sum_{i=1}^N \log q_\phi(\epsilon_i | \theta_i).$$

2. Model Architecture and Training Procedure

RNLE uses a Masked Autoregressive Flow (MAF) neural density estimator, as implemented in the sbi package. The residual likelihood $q_\phi(\epsilon|\theta)$ is constructed by composing $K$ autoregressive (invertible) transforms:

$$\epsilon \longleftrightarrow u = f_K \circ \cdots \circ f_1(\epsilon; \theta), \quad u \sim \mathcal{N}(0, I),$$

where each transform $f_k$ is parameterized by an ARNN (e.g., MADE) that conditions on $\theta$ and the preceding elements of $\epsilon$. The paper adopts an architecture of five flows ($K=5$), with each ARNN having two hidden layers of 50 units and ReLU activations.

Training is based solely on pure-noise residuals $\epsilon_i$, optionally paired with corresponding parameter values $\theta_i$ (used, for instance, to encode PSD or whitening information with real detector data). Crucially, no signal is injected during training. For real data, 1 s segments of whitened detector data are used, introducing a scale parameter $\sigma \sim \mathrm{Uniform}[0, 2]$ during whitening. Non-Gaussian noise artifacts (glitches) are incorporated by sampling corresponding data segments during training, which allows the model to learn heavy-tailed residual distributions.

Optimization employs Adam with a default learning rate of $10^{-3}$, batch size $\sim 256$, and training durations of $5 \times 10^4$ to $1 \times 10^5$ steps, managed by the sbi toolkit (Emma et al., 20 Jan 2026).

At inference, the workflow for evaluating $p(d|\theta)$ under the learned model is:

1. Generate $m(\theta)$ for the proposed $\theta$.
2. Whiten and subtract $m(\theta)$ from observed $d$, yielding residual $\epsilon = d - m(\theta)$.
3. Evaluate $q_\phi(\epsilon|\theta)$ for likelihood computation.

3. Advantages over Standard Neural Likelihood Estimation (NLE)

RNLE offers several key improvements over traditional NLE approaches:

• Dimensionality Reduction: Standard NLE seeks to learn $p(d|\theta)$ across the simulator's full output dimension, which includes both signal and noise. RNLE only models $p(\epsilon|\theta)$, typically a lower-dimensional space—especially when $\theta$ may parameterize only the noise PSD. This results in reduced simulation requirements for accurate likelihood learning.
• Non-Gaussian Robustness: By modeling the true (possibly glitch-dominated) noise distribution empirically, RNLE circumvents the limitations of the Whittle likelihood, which assumes Gaussianity and stationarity. RNLE accurately learns even heavy-tailed noise structures directly from data, yielding robustness to real-world non-Gaussian artifacts.
• Empirical Efficiency and Calibration: In a sine-Gaussian toy model, RNLE attains Jensen-Shannon divergence $< 3 \times 10^{-3}$ with $\sim 10^3$ simulations, while standard NLE requires an order of magnitude more. For 10D and 15D binary black hole (BBH) injections, RNLE achieves JS divergences $< 0.01$ with $N_\mathrm{sim}\approx 3 \times 10^4$. In tests with strong glitches, RNLE posteriors remain unbiased (true parameter contained in 90% credible interval), in stark contrast to biased Whittle likelihood inferences (Emma et al., 20 Jan 2026).

4. Performance Evaluation and Applications

Performance of RNLE is validated across multiple domains:

• Toy Models: For sine-Gaussian signals with parameters $\{f,\,A,\,\alpha\}$ plus scale noise, RNLE achieves convergence in both 2D and 4D parameter spaces with $\sim 10^3$ simulations, measured via JS divergence of posterior samples.
• Simulated Gravitational-Wave (GW) Signals: In 1D chirp-mass benchmarks, RNLE/Whittle JS divergence is $< 0.003$ for $N_\mathrm{sim} \gtrsim 2\times 10^3$; posterior probability (PP) test p-values are $\sim 0.1$ (indicative of correct coverage). High-dimensional BBH cases—10D, 11D, and 15D—exhibit JS$<0.01$ for all intrinsic parameters and slightly higher JS for extrinsic angles, within expected sampling variability.
• Real Detector Noise: For quasi-Gaussian 1 s segments from LHO (Aug 2019), RNLE and Whittle posteriors on injected BBH signals yield JS$\approx$0.01 with calibration passed. Under loud glitches (SNR $>40$), RNLE recovers true chirp mass within the 90% credible interval (zero offset condition); Whittle likelihood suffers catastrophic bias. With blip glitches (O2, SNR $\sim 15$), RNLE—trained on matched-glitch datasets—produces unbiased posteriors irrespective of offset, unlike BayesWave deglitching which fails near glitch time (Emma et al., 20 Jan 2026).
• Ensemble Weighting: When multiple independently trained $q_{\phi_i}$ models are available, mixture posteriors can be formed:

$$p(\theta | d) = \sum_{i=1}^M w_i p_i(\theta | d), \quad w_i = \frac{\mathcal{Z}_i}{\sum_j \mathcal{Z}_j}$$

Here, $\mathcal{Z}_i$ is the measured evidence for the $i$-th model. While evidence-based weights are intuitive, in glitch-dominated regimes evidence may sometimes favor a biased realization. Nevertheless, variability in evidence acts as a sensitive indicator of density estimator convergence.

5. Implementation and Software

RNLE is implemented in the sbilby package, which extends the Bilby inference library. The basic workflow is:

pip install sbilby from sbilby import RNLELikelihood, RNLEConfig config = RNLEConfig(noise_model=..., waveform_model=...) lnL = RNLELikelihood(config) result = bilby.run_sampler(likelihood=lnL, priors=..., sampler='dynesty')

All data preprocessing steps—whitening, windowing, PSD estimation—are handled internally to match Bilby’s Whittle likelihood conventions. The implementation is designed for seamless integration in pipeline-driven scientific inference and supports rapid deployment for gravitational-wave astronomy (Emma et al., 20 Jan 2026).

6. Broader Applicability and Prospects

RNLE’s methodology is directly transferable to a range of domains characterized by additive deterministic signals and complex, possibly non-Gaussian noise. Representative application areas include:

• Radio or X-ray pulsar timing under non-stationary terrestrial or astrophysical transients
• Seismic signal analysis with environmental noise artifacts
• Biomedical time series (e.g., ECG/EEG with movement artifacts)
• Particle physics scenarios with rare signal peaks plus detector backgrounds

By decoupling signal modeling from noise likelihood estimation, and directly learning the true residual noise from data, RNLE offers robust parameter inference under real-world noise violations—particularly when noise structure is not amenable to analytic modeling or stationary approximations. This approach permits orders-of-magnitude improvements in simulation efficiency while maintaining the rigor of Bayesian inference. In summary, RNLE establishes a principled, scalable, and empirically validated framework for simulation-based likelihood estimation where noise realism and robustness are essential (Emma et al., 20 Jan 2026).