Adaptive Online Emulation (AOE)

• Adaptive Online Emulation (AOE) is a framework that incrementally builds surrogate models using online learning, active sampling, and uncertainty quantification.
• It employs methods like Gaussian Processes, PCA, and OS-ELMs to approximate and update expensive simulation outputs efficiently.
• AOE accelerates computational tasks in fields like cosmology and remote sensing by reducing direct model evaluations and integrating seamlessly with inference pipelines.

Adaptive Online Emulation (AOE) refers to a class of algorithms and software frameworks designed to construct and update surrogate models for expensive computational codes or simulations in an online and adaptive manner. Instead of relying on extensive offline training over wide parameter regimes, AOE methods incrementally learn and refine surrogates as new data are acquired, targeting only the regions of the parameter space that are being actively explored. This approach has proven highly effective in accelerating inference and simulation tasks in domains such as cosmology, remote sensing, and physical simulation modeling by reducing the number of expensive code evaluations required while maintaining rigorous statistical accuracy.

1. Core Algorithmic Principles

Adaptive Online Emulation algorithms integrate online learning, active sampling, and uncertainty quantification. The central operational premise is to alternate between sampling the true (expensive) model and fitting a fast statistical surrogate, such as a Gaussian Process (GP) or an online-trained neural network, in regions of interest. Emulation proceeds iteratively:

• Maintain a cache of support points comprising evaluated parameter-vector/result pairs.
• Fit or update the surrogate model (e.g., GP or Online Sequential Extreme Learning Machine) to the current cache.
• Estimate the surrogate's error at candidate points using automatic error metrics (typically via predictive variance).
• Selectively accept surrogate predictions when deemed accurate; otherwise, execute the true model, update the cache, and retrain the surrogate.
• Concentrate training data and modeling efforts in high-likelihood or high-gradient areas determined by the sampler or acquisition function (Günther et al., 17 Mar 2025, Tahseen et al., 11 Aug 2025, Svendsen et al., 2019).

2. Surrogate Model Construction

Surrogate modeling within AOE leverages flexible statistical models with tractable error estimates:

• Gaussian Processes and PCA: In high-dimensional output settings (e.g., power spectra or multi-band reflectance data), outputs are first compressed via weighted Principal Component Analysis (PCA), and each PCA coefficient is modeled by an independent 1D GP. The GP kernel typically combines anisotropic squared-exponential (RBF), linear, and white noise components, with hyperparameters fitted by maximizing marginal likelihood (Günther et al., 17 Mar 2025).
• Online Sequential ELMs (OS-ELM): For time-stepping simulation frameworks, OS-ELMs provide a numerically stable, incremental learning scheme for neural surrogates. Input–output batches are mapped to single-hidden-layer architectures with fixed random weights; output weights are updated by accumulating sufficient statistics and infrequent (but stable) ridge regression solves (Tahseen et al., 11 Aug 2025).
• Multi-Output GP Models: When simultaneous prediction of multiple outputs is needed, independent GPs can be trained per output dimension, with predictive means and variances aggregated to characterize overall uncertainty (Svendsen et al., 2019).

The predictive variance from these surrogates naturally provides an error bar at each test point, enabling rigorous, data-driven selection of when the emulator is trustworthy.

3. Active and Adaptive Sampling Strategies

A distinguishing feature of AOE is its explicit use of uncertainty for active, targeted data acquisition:

• Acquisition Functions: Informative points are selected according to acquisition functions constructed from combinations of predictive variance ("diversity") and gradient magnitude ("geometry") terms (Svendsen et al., 2019). Example forms include products or sums of output-wise variances and gradients, modulated by a time-dependent parameter $\beta_t$ to balance exploration and exploitation over the course of data acquisition.
• Automated Acceptance Decision: An explicit, user-configurable precision function (e.g., involving log-likelihood differences) governs the maximum acceptable error for surrogate predictions at each sampler proposal. Only when predicted error is below this threshold does the AOE pipeline accept the emulator's output; otherwise, a ground-truth evaluation is performed and the surrogate is updated (Günther et al., 17 Mar 2025).

This framework ensures computational resources are focused where model uncertainty is largest or the statistical objective is most sensitive, producing compact and expressive training datasets and surrogates.

4. Integration with Scientific Inference and Simulation Pipelines

AOE has been specifically engineered to interface seamlessly with existing scientific codebases and inference workflows:

• Sampler Interfacing: Surrogates are deployed as drop-in replacements for model evaluations inside loops of samplers such as Metropolis–Hastings or Hamiltonian Monte Carlo, as in Cobaya and MontePython (Günther et al., 17 Mar 2025).
• Physical Model Wrapping: For time-evolution simulators, the emulator is called in place of the computational module (e.g., a radiative transfer solver), with control logic swapping between the true model and the surrogate based on accuracy estimates (Tahseen et al., 11 Aug 2025).
• Differentiability: Architectures implemented with JAX ensure all components, including emulation, likelihood, and sampling routines, are differentiable. This facilitates gradient-based model search, BFGS minimization, and efficient NUTS-type sampling for high-dimensional Bayesian inference (Günther et al., 17 Mar 2025).

Integration requires minimal changes to external code: auxiliary wrappers or plugin hooks are used to orchestrate emulator calls versus ground-truth evaluations.

5. Empirical Performance and Use Cases

AOE has achieved substantial acceleration and maintained statistical accuracy across multiple scientific domains:

• Cosmological Parameter Inference: Using OLÉ (Günther et al., 17 Mar 2025), speed-ups of $30\times$ to $350\times$ over direct calls to Einstein–Boltzmann solvers have been demonstrated, with effective sampling rate (ESS/CPU-hour) increases from 0.35 to 17 and posterior shifts below $0.05\sigma$. Differentiable emulation combined with candl yields an additional $4\times$ gain, enabling efficient NUTS sampling.
• Large-Scale Physical Simulations: In a 1D atmospheric model (GJ1214b), AOE with OS-ELM achieves $11.1\times$ speedup (91% time reduction) over 200,000 timesteps, with mean absolute percentage errors below 0.01% and a drastic reduction in required training samples ($10^4$ vs $10^6$–$10^8$ for deep learning surrogates) (Tahseen et al., 11 Aug 2025).
• Remote Sensing Emulation: Active multi-output GP emulators outperform sequential Sobol/LHS acquisition methods in constructing surrogates for radiative-transfer codes and related black-box, multi-output simulations (Svendsen et al., 2019).

Summary Table: Representative Metrics from OLÉ in Cosmological Inference

Use Case (Data/Model)	Speed-up Factor	Posterior Shift
ΛCDM + Planck2018 + Pantheon + BAO	~100×	< 0.03σ
ΛCDM + Ωk + w₀–wₐ + Σmν + DESI	~350×	< 0.05σ
Euclid Stage IV (10 parameters)	~50×	—
NEDE model (non-Gaussian posteriors)	~30×	Stable

These figures are based on effective sample size per CPU-hour, after accounting for all data acquisition and training overheads (Günther et al., 17 Mar 2025).

6. Limitations and Prospective Extensions

Despite its substantial empirical successes, AOE is subject to limitations:

• Scalability: Retraining PCA and GP surrogates becomes expensive as the support cache exceeds $O(10^3)$ points; practical implementations cap the cache at $30\times$0 (Günther et al., 17 Mar 2025).
• Output Size and Format: Current designs target fixed-size outputs; emulating variable resolution predictions is not natively supported.
• Acquisition Function Simplicity: Thresholded variance-based acceptance rules could be refined using more advanced active learning criteria, such as expected improvement or information-theoretic metrics (Günther et al., 17 Mar 2025, Svendsen et al., 2019).
• Nonstationarity and Roughness: Gradient-based acquisitions presume output smoothness; nonstationary or rough target functions may require alternative kernels, such as Matérn or piecewise models (Svendsen et al., 2019).
• Parallelism and Nested Sampling: Native batch/sequential acquisition for parallel simulators, and full support for nested sampling, are under development.
• Multifidelity and Dynamic Models: Extensions to multifidelity emulation, and emulation of dynamic time-series outputs (e.g., via state-space GPs or multiple OS-ELMs), are indicated as promising future developments (Tahseen et al., 11 Aug 2025, Svendsen et al., 2019).

AOE provides a modular, extensible approach to active surrogate modeling, accelerating fundamental tasks in scientific computing while maintaining strict control over emulator accuracy and resource expenditure.