Adaptive On-the-Fly Learning Scheme

• On-the-fly learning scheme is an adaptive computational method that updates model components dynamically, eliminating full offline training and reducing memory usage.
• It employs active and incremental feedback, triggering recalibration or data acquisition based on error and uncertainty for enhanced scalability and efficiency.
• Widely applicable in quantum simulations and atomistic modeling, the scheme optimally manages large combinatorial datasets with real-time updates.

An on-the-fly learning scheme refers to an adaptive computational or machine learning methodology where model components, predictions, or structural updates are performed dynamically in response to data as it arrives, rather than via static offline construction. In the context of numerical simulation, applied machine learning, and scientific computing, this often enables substantial reductions in computational cost, improved scalability, and broader applicability to data- or configuration-rich domains that would otherwise be intractable due to memory or processing constraints.

1. Fundamental Principles and Motivation

On-the-fly learning schemes are characterized by their ability to adapt to evolving input without requiring precomputed models, full storage of all configurations, or retraining from scratch. Typical motivating factors include:

• Avoidance of prohibitive memory/storage costs: Rather than storing large objects such as dense matrices, state vectors, or reference datasets, on-the-fly methods generate, update, or refine model components as needed.
• Operational adaptability: The system is able to respond to new data or configurations without waiting for a “full batch” or a static data regime.
• Active/incremental feedback: Prediction errors, uncertainties, or state novelty trigger focused recalibration or data acquisition, ensuring relevance and robustness.

In scientific simulation, this paradigm is especially pertinent when model spaces (e.g., quantum Hamiltonian matrices or atomic environments) are combinatorially large, and in engineering contexts where the operational domain or data generating process is non-stationary.

2. Matrix-Free Diagonalization in Quantum Models

A canonical instantiation of on-the-fly learning is found in the matrix-free exact diagonalization (ED) of quantum electronic Hamiltonians (Kashin et al., 2015). Here, the principal computational object is the Hamiltonian matrix, often of dimension $D \gg 10^9$, arising, for example, in Anderson impurity models used for dynamical mean-field theory (DMFT).

Key algorithmic innovation:

• On-the-fly regeneration of sparse matrices: Rather than storing even the nonzero elements (as would be done with compressed row storage), at each iteration of the Lanczos or Arnoldi scheme, the nonzero Hamiltonian entries relevant to the current state vector are computed dynamically using the occupation number representation and known interaction parameters.
• Efficient representation of the many-body basis: Only unique occupation number configurations for fixed particle count and spin are kept in memory, and all full-state representations are generated implicitly as needed through efficient enumeration schemes.

This results in:

• Memory reduction: Up to $85-87\%$ compared to traditional CRS storage in large-scale cases (e.g., 18 effective orbitals corresponding to $D = 2.4 \times 10^9$).
• Scalability: Matrix-vector multiplications are efficiently handled in a distributed memory setting. Communication is limited to the “active” segments of the state vector, minimizing data transfer.

Applicability: The method is effective for partial spectrum computations (ground/low-lying states), multi-orbital DMFT solvers, and other strongly correlated electron systems where combintorial Hilbert space growth is a critical bottleneck.

3. Active Learning with Uncertainty Quantification

Adaptive, on-the-fly model refinement is also central to modern atomistic simulation workflows, especially when interfacing machine learning surrogates with high-fidelity quantum-mechanical calculations (Vandermause et al., 2019, Volkmer et al., 8 Aug 2025).

Core workflow:

• Surrogate models (e.g., Gaussian process regression): Employed to predict atomic energies and forces based on descriptors derived from the current atomic environment.
• On-the-fly error/uncertainty monitoring: For a given environment, the epistemic (model) uncertainty is estimated analytically.
• Triggering data acquisition: If the predicted uncertainty for a new structure exceeds a threshold, a reference calculation (typically DFT) is automatically performed. The resulting data is injected into the model, which is re-trained, often via Bayesian linear regression (kernel-based MLIPs) or deep learning (e.g., MACE architectures).
• Adaptive, self-improving dataset: Only those atomic environments that are both likely to be visited and for which the model is uncertain are added to the training set, preventing redundant or irrelevant computation.

Specifics for interatomic potential learning:

• Energy model:

$$U = \sum_{i=1}^{N_A} U_i = \sum_{i=1}^{N_A} \sum_{i_B=1}^{N_B} w_{i_B} K(x_i, x_{i_B})$$

where $x_i$ are local atomic descriptors, $K$ is a kernel (e.g., a polynomial kernel of exponent $\zeta$), and $w_{i_B}$ are weights fit by Bayesian regression.

• Uncertainty quantification: The posterior predictive variance yields an error bar per force (or energy), optimizing the balance between exploration and efficiency.

Outcome: Empirical studies show that over $97-98\%$ of DFT calls can be avoided during training through judicious uncertainty-based triggering (Volkmer et al., 8 Aug 2025). Final machine-learned potentials closely match experimental elastic constants of Al-Mg-Zr alloys with deviations of only a few GPa.

4. Scalability and Computational Efficiency

Memory and computational efficiency are a defining feature across on-the-fly schemes:

Feature	Traditional Approach	On-the-fly Scheme
Storage	Complete or CRS matrix, large fixed workspaces	Memory scales with unique configurations or active region only
Data Acquisition	Precompute or pre-label entire datasets	Triggered adaptively via errors/uncertainty
Parallelism	Uniform or static batch assignment	Dynamic redistribution, only “active” indices participate

In distributed computation, on-the-fly approaches minimize communication and focus node resources only where computation is locally required. In machine-learned potential construction, only the minimal training set needed to explain the current trajectory is maintained—scaling enables simulation of larger systems or more complex physics in multiscale models.

5. Generalization and Broader Applicability

On-the-fly learning schemes broadly apply to:

• Quantum many-body diagonalization and DMFT: Enables treatment of large, correlated systems previously inaccessible to exact diagonalization (Kashin et al., 2015).
• Atomistic force field construction: Active learning frameworks with Bayesian error control for rapid, interpretable force fields applicable across rare-event–dominated dynamics (Vandermause et al., 2019, Volkmer et al., 8 Aug 2025).
• Other computational physics domains: Any workflow where the object of interest (e.g., matrix, potential, classifier, label) grows faster than available memory or where new configurations arise dynamically, can in principle benefit from on-the-fly representations, regeneration, or adaptation.

Potential extensions include integration with hybrid CPU/GPU architectures, seamless multiscale coupling, and real-time adaptive modeling for experimental or industrial processes.

6. Future Perspectives and Theoretical Implications

The on-the-fly paradigm foreshadows a general trend in computational science and machine learning toward adaptivity, judicious memory management, and dynamic error control. In quantum simulation, it expands the scope of treatable problems by orders of magnitude in Hilbert space dimension. In materials modeling, it accelerates exploration and design in complex compositional spaces and heterogeneous microstructures.

A plausible implication is that as models and experiments become more tightly coupled and data streams evolve, on-the-fly learning principles will become foundational to robust, scalable, and interpretable scientific computing workflows.