On-the-Fly Learning: Adaptive, Real-Time Methods

• On-the-fly learning is a paradigm where model updates, data acquisition, and inference occur simultaneously during deployment, enabling continuous adaptation.
• It leverages algorithmic patterns such as streaming updates, adaptive supervision, and dynamic reweighting to efficiently operate under latency, memory, or cost constraints.
• Empirical studies show that on-the-fly methods can significantly reduce computational costs and improve real-time performance in fields like robotics, quantum chemistry, and computer vision.

Searching arXiv for recent and foundational papers on “on-the-fly learning” to ground the article in published work. On-the-fly learning denotes a family of learning paradigms in which model construction, adaptation, or data acquisition occurs during deployment, streaming interaction, or iterative computation rather than exclusively in a fully precomputed offline phase. Across machine learning, control, robotics, scientific computing, computer vision, and natural-language processing, the term is used for related but non-identical mechanisms: online parameter updates from sequential data, adaptive querying of expensive labels or simulators, test-time policy adaptation, dynamic generation of synthetic or auxiliary training signals, and prediction from partial sequences before complete observations are available. The common motif is that learning or model refinement is interleaved with operation under latency, memory, or cost constraints, often with the objective of avoiding full retraining, reducing redundant computation, or responding to nonstationary conditions (Vinod et al., 1 Jun 2026).

1. Conceptual scope and defining characteristics

In the literature, on-the-fly learning is not a single algorithmic template but a recurrent operational pattern. One instance is streaming supervision, where labeled information is acquired selectively as examples arrive one by one, as in data-stream classification with active learning budgets and class-balanced memory (Malialis et al., 2022). Another is continual or trial-by-trial learning, where each newly observed example immediately updates a model, as in the fruit-fly-inspired associative rule that modifies only synapses feeding the target output neuron (Shen et al., 2021). A third is adaptive resource allocation, where the learning system decides during execution which fidelity of oracle or simulator to query next, as in adaptive multifidelity quantum-chemistry learning (Vinod et al., 1 Jun 2026).

The same phrase also denotes prediction or control without waiting for complete inputs. In facial-expression recognition, a CNN+LSTM architecture is trained so that partial sequences are already discriminative, enabling prediction as frames stream in rather than after explicit temporal segmentation (Baddar et al., 2017). In active flow control, a learned flow map for drag and lift is used online so that control optimization proceeds without forward CFD solves during deployment; in that work, the model parameters remain fixed during control, and “on-the-fly” refers to instantaneous online use of the surrogate rather than online retraining (Liu et al., 8 Mar 2026).

A further variant is test-time adaptation. Vision-language-action models can update their policy parameters within an episode using dense progress signals, starting from a pretrained policy and optimizing online at inference (Liu et al., 11 Jan 2026). In noisy-label learning, sample weights are learned in real time from model-derived features, so that the training process itself continuously re-estimates which examples should contribute more or less to gradient updates (Heck et al., 2024).

These usages suggest a broad definition: on-the-fly learning is learning under operational coupling, where data arrival, inference, control, or expensive supervision and model updates are entangled in time rather than cleanly separated into offline training and frozen deployment.

2. Core algorithmic patterns

Several recurring algorithmic patterns organize the field.

Streaming update of model parameters is the most direct form. Online Deep Learning formulates a data-stream setting in which at each round $t$ the learner receives $x_t$, predicts $\hat y_t$, then observes $y_t$, with a hedged ensemble $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$ over multiple depths and multiplicative updates of $\alpha^{(l)}$ to adapt effective depth over time (Sahoo et al., 2017). The fruit-fly model is even more local: sparse high-dimensional expansion is followed by a rule that only potentiates synapses from active Kenyon cells to the target mushroom-body output neuron, with other weights frozen or subject only to slight decay (Shen et al., 2021).

Adaptive acquisition of supervision or oracle calls is another major pattern. In Augmented Queues, an active learning budget $B$ constrains how often labels may be queried in a stream, and training occurs only after a queried example is inserted into class-specific FIFO queues and augmented on the fly (Malialis et al., 2022). In adaptive-MFML, the system monitors the mean absolute error on a small highest-fidelity validation set and uses the local improvement criterion $\Delta_f=E_{prev}-E_{curr}$ to decide when to stop sampling at fidelity $f$ and step up to $f+1$ (Vinod et al., 1 Jun 2026).

Dynamic reweighting rather than relabeling or filtering appears in noisy-label learning. STORM defines a learnable rescaling function $x_t$0, where $x_t$1 is computed on the fly from the model being trained, and uses these weights in both the inner training loss and the outer meta-objective sampled from the same noisy corpus (Heck et al., 2024). This replaces a binary keep-or-discard decision with continuous sample weighting.

Online control with learned surrogates forms a distinct pattern. Flow Map Learning constructs a discrete-time memory-augmented map $x_t$2 from histories of quantities of interest and controls to the next drag and lift values, after which DRL or MPC can be performed cheaply in the surrogate (Liu et al., 8 Mar 2026). In safe control under model uncertainty, a Gaussian-process model and a control barrier function are coupled with a bandit-style exploration trigger: if the safety filter becomes infeasible, an exploration action maximizing an upper confidence bound is applied, new data are collected, and the GP is updated until feasibility is restored (Capone et al., 12 Jun 2025).

Learning from partial or auxiliary information is also common. Rare-word embeddings can be computed from spelling or dictionary definitions through a small encoder $x_t$3 trained end-to-end with the downstream task, so that a useful embedding is generated at lookup time rather than pre-stored for every word type (Bahdanau et al., 2017). In partial-sequence expression recognition, frame-level intensity targets are estimated from similarity to apex features and used to shape features that support early-time prediction (Baddar et al., 2017).

3. Data, memory, and supervision regimes

A central driver of on-the-fly learning is constrained access to labels, storage, or high-quality targets.

In stream classification, the memory architecture can be explicit and balanced. Augmented Queues maintains $x_t$4 FIFO queues of fixed capacity $x_t$5, one per class, so that memory is limited to $x_t$6 samples while preserving class-wise balance (Malialis et al., 2022). Synthetic samples are not stored persistently; they are generated only at training times, which the method identifies as a major advantage because no additional memory space is reserved for augmentation.

In continual learning, memory may instead be implicit in parameters and architectural sparsity. The fly-inspired model uses fixed random sparse expansion plus winner-take-all coding to reduce interference between classes, so that online updates can proceed without replay or external memory (Shen et al., 2021). The Perpetual Learning Machine pushes this logic further by discarding the original dataset after initial training and relying on a self-supervised loop between a storage network and a recall network in Perpetual Stochastic Gradient Descent; memory resides in the joint weights rather than in a replay buffer (Simpson, 2015).

In scientific ML, the scarce resource is often not labels per se but expensive oracle evaluations. Adaptive-MFML defines per-fidelity datasets $x_t$7, where $x_t$8 is a quantum-chemistry oracle at fidelity $x_t$9, and imposes a nested-subset condition $\hat y_t$0 (Vinod et al., 1 Jun 2026). The algorithm’s objective is to meet an error target while minimizing the total cost $\hat y_t$1.

In embedded robotics, the supervision regime may be self-supervised because external labels are unavailable in the field. On a nano-drone, 512 images are acquired on board and fine-tuning uses an ego-motion consistency loss $\hat y_t$2 with $\hat y_t$3, where in self-supervised mode the subject motion is assumed static over short intervals (Cereda et al., 2024). This is paired with stringent hardware constraints: the 512-image on-device training procedure requires only $\hat y_t$4MB of memory and can consume as low as $\hat y_t$5mW or run in $\hat y_t$6ms at $\hat y_t$7mW.

These examples show that “on-the-fly” often arises when one cannot assume simultaneous abundance of labels, memory, and compute. The learning protocol is therefore co-designed with the bottleneck: query budgets, sparse updates, transient augmentation, self-supervision, or adaptive oracle selection.

4. Representative formulations and mechanisms

The mathematical forms used in on-the-fly learning vary by domain, but several mechanisms recur.

In adaptive multifidelity learning, the predictor for the highest-fidelity property takes a telescoping form,

$\hat y_t$8

with $\hat y_t$9, and fidelity switching is governed by whether the local improvement $y_t$0 falls below a tolerance $y_t$1 (Vinod et al., 1 Jun 2026). The stopping rule couples accuracy saturation directly to data-generation cost.

In online deep learning, the multi-depth network is trained by a combined loss across depth experts, while Hedge-style multiplicative weights update the coefficients $y_t$2. This yields a soft model-selection mechanism in which shallow experts dominate early and deeper experts can take over later as more data arrive (Sahoo et al., 2017). An analogous expert-weight mechanism appears in online adaptive metric learning, where embedding experts of increasing depth are combined with weights $y_t$3, updated by an exponential Hedge rule, while each expert is trained with the Adaptive-Bound Triplet Loss (Gao et al., 2021).

In noisy-label learning, the critical object is not the classifier alone but the rescaler. STORM computes sample features from repeated stochastic forward passes and updates the rescaler parameters $y_t$4 through an outer gradient and a backward-through-backward meta term, using noisy minibatches for both inner and outer loops (Heck et al., 2024). In effect, the training algorithm learns how to modulate its own loss landscape during optimization.

In test-time reinforcement learning for vision-language-action models, the reward is derived from progress differences,

$y_t$5

where $y_t$6 is estimated by a frozen progress predictor. The policy then performs value-free PPO-style clipped updates online, with one-step advantage $y_t$7 (Liu et al., 11 Jan 2026). This makes the adaptation signal dense and immediately available during inference.

In streaming label learning for multi-label problems, the arrival of a new label is handled in two steps: a sparse self-representation

$y_t$8

followed by a regularized ERM for the new classifier $y_t$9 that stays close to the inherited prior $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$0 (You et al., 2016). Here, on-the-fly learning refers to modeling newly arrived labels without retraining the full multi-label system.

In rare-word NLP, the auxiliary embedding module computes $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$1 from spelling or definitions and combines it with the stored head-word embedding via $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$2 or $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$3 (Bahdanau et al., 2017). This is a form of instance-conditional representation construction at inference and training time.

5. Empirical regimes and observed benefits

Reported gains from on-the-fly learning are domain-specific, but the literature repeatedly associates the paradigm with reduced cost, improved adaptability, or earlier useful prediction.

In quantum chemistry, adaptive-MFML reports that the adaptive algorithm reduces data generation costs by up to a factor of $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$4 compared to single fidelity methods and improves upon standard MFML by up to a factor of $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$5 (Vinod et al., 1 Jun 2026). On the VIB5 PES benchmark at target MAE $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$6 kcal/mol, the reported costs are $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$7 h for single-fidelity KRR, $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$8 h for fixed MFML, and $F(x)=\sum_{l=0}^L \alpha^{(l)} f^{(l)}(x)$9 h for adaptive-MFML. On ANI-1ccx, reaching MAE $\alpha^{(l)}$0 kcal/mol requires $\alpha^{(l)}$1 h for single-fidelity KRR, $\alpha^{(l)}$2 h for fixed MFML, and $\alpha^{(l)}$3 h for adaptive-MFML.

In active flow control, a learned surrogate enables optimization without forward simulations during control. Reported closed-loop performance includes $\alpha^{(l)}$4 drag reduction for DRL-PPO+FML-1 and $\alpha^{(l)}$5 reduction for MPC+FML-1 at $\alpha^{(l)}$6, while replacing the cost of $\alpha^{(l)}$7 forward Navier–Stokes solves, estimated as $\alpha^{(l)}$8 CPU sec, with surrogate rollout and optimization costing $\alpha^{(l)}$9 sec (Liu et al., 8 Mar 2026).

In data-stream classification, Augmented Queues with small memory and on-the-fly augmentation can match or exceed much larger non-augmented memory. On MNIST with $B$0 minority under $B$1, the reported prequential G-mean rises from $B$2 to $B$3 with augmentation, versus $B$4 for non-augmented memory $B$5 (Malialis et al., 2022). Increasing $B$6 from $B$7 to $B$8 boosts G-mean by $B$9 points, but at the cost of $\Delta_f=E_{prev}-E_{curr}$0 memory.

In on-device self-supervised nano-drone adaptation, the best SSL fine-tune-fc configuration reduces $\Delta_f=E_{prev}-E_{curr}$1 by $\Delta_f=E_{prev}-E_{curr}$2 versus the simulation baseline, with only $\Delta_f=E_{prev}-E_{curr}$3 kB extra memory and $\Delta_f=E_{prev}-E_{curr}$4 M MAC/step, while the full training procedure is deployed aboard a GAP9 SoC under strict power and memory constraints (Cereda et al., 2024).

In continual learning, the fly-inspired online rule substantially outperforms several continual-learning baselines. On MNIST-20 after 5 tasks, reported test accuracies are FlyModel $\Delta_f=E_{prev}-E_{curr}$5, BI-R $\Delta_f=E_{prev}-E_{curr}$6, GEM $\Delta_f=E_{prev}-E_{curr}$7, EWC $\Delta_f=E_{prev}-E_{curr}$8, and Vanilla $\Delta_f=E_{prev}-E_{curr}$9; average memory-loss is $f$0 for FlyModel versus $f$1 for GEM and $f$2 for EWC (Shen et al., 2021).

In noisy-label NLP, STORM improves the average overall metric across five classification tasks with $f$3 noise from $f$4 without rescaling to $f$5, and improves MultiWOZ joint goal accuracy from $f$6 to $f$7 on real noisy data (Heck et al., 2024).

These results do not establish a single universal advantage of on-the-fly learning; rather, they indicate recurring empirical payoffs when the learning protocol is matched to the deployment bottleneck.

6. Limitations, misconceptions, and points of variation

A common misconception is that on-the-fly learning always implies online parameter adaptation. The flow-control literature provides a clear counterexample: in Liu et al., all DNN parameter training is done offline, the FML model is frozen during deployment, and “on-the-fly” means that the feedback control law can be computed instantly via the learned surrogate without additional CFD solves (Liu et al., 8 Mar 2026). Conversely, TT-VLA explicitly updates policy parameters within an episode at test time (Liu et al., 11 Jan 2026). The phrase therefore covers both online learning and online use of prelearned surrogates.

Another misconception is that on-the-fly methods eliminate all prior training. Many methods are hybrid. Rare-word embedding modules are trained end-to-end with downstream objectives but still rely on an in-vocabulary embedding table and auxiliary resources such as spelling or dictionary definitions (Bahdanau et al., 2017). Nano-drone adaptation begins from a pretrained pose-estimation CNN and fine-tunes only selected components on device (Cereda et al., 2024). Adaptive-MFML begins with initial random points at each fidelity and user-defined tolerances $f$8 and $f$9 (Vinod et al., 1 Jun 2026).

Limitations are likewise heterogeneous. Sequential on-demand quantum-chemistry workflows can under-utilize parallel resources, whereas fixed MFML can evaluate all fidelities in parallel (Vinod et al., 1 Jun 2026). Augmented Queues requires at least $f+1$0 initial labeled samples per class and does not address concept drift explicitly (Malialis et al., 2022). TT-VLA depends on the quality of the frozen progress predictor $f+1$1; if progress estimates are noisy or the task lacks monotone progress, the reward can mislead adaptation (Liu et al., 11 Jan 2026). Safe bandit exploration assumes GP confidence bounds, sampling-time conditions, and CBF feasibility margins, and exploration inputs may be aggressive even if optimism is safety-guided (Capone et al., 12 Jun 2025).

Several methods also reveal a trade-off between adaptability and stability. The fruit-fly results show that decreasing weights to other classes can accelerate forgetting when class representations overlap, whereas the variant that only increases correct-class weights performs best in continual learning (Shen et al., 2021). STORM shows that continuous rescaling is superior to hard filtering; the reported binary rescaling ablation drops to $f+1$2 average, far below the continuous method (Heck et al., 2024). These findings suggest that many successful on-the-fly systems regulate update locality and magnitude as carefully as they regulate what data to acquire.

7. Relation to adjacent areas and likely directions

On-the-fly learning overlaps with online learning, continual learning, active learning, test-time adaptation, self-supervised adaptation, multifidelity modeling, streaming control, and TinyML, but it is not reducible to any one of them. Online deep learning emphasizes one-pass adaptation under concept drift with regret guarantees against the best fixed depth in hindsight (Sahoo et al., 2017). Streaming label learning focuses on newly arriving labels rather than newly arriving instances (You et al., 2016). Adaptive multifidelity learning treats data acquisition cost as the principal online variable (Vinod et al., 1 Jun 2026). Safe bandit exploration joins uncertainty-aware model learning to safety certificates so that additional data are collected specifically when the safety filter is infeasible (Capone et al., 12 Jun 2025).

Several extensions recur across papers. Adaptive-MFML explicitly identifies active-learning scores such as KRR-predictive variance, differentiable optimization of $f+1$3, and hybrid MLIP$f+1$4QC or MLIP$f+1$5MFML pipelines as next steps (Vinod et al., 1 Jun 2026). Augmented Queues points to drift detection, selective memory replacement, latent-space augmentation, and budget-aware curricula (Malialis et al., 2022). STORM indicates that richer per-sample features and class-specific rescalers become especially important under real and imbalanced noise (Heck et al., 2024). OAML suggests integration with convolutional or recurrent feature extractors and richer hedging strategies (Gao et al., 2021). Nano-drone self-supervision is presented as extensible to depth, flow, object detection, and architectures such as ResNet-like models and transformers on GAP-class SoCs or MCUs with more than $f+1$6 MB RAM and SIMD/FPU support (Cereda et al., 2024).

A plausible implication is that future on-the-fly systems will be increasingly co-designed across four coupled axes: when to update, what to query, how much to trust newly acquired signals, and which parts of the model are permitted to change. The literature already spans fully local associative rules, expert-hedged online deep networks, meta-learned sample reweighting, safety-triggered exploration, and hardware-aware self-supervised fine-tuning. Taken together, these works define on-the-fly learning less as a narrow subfield than as a deployment-centric design principle for machine learning systems operating under sequential information, scarce supervision, and evolving environments.