Learning While Deploying (LWD)

• Learning While Deploying (LWD) is a deployment-centered design principle where systems adapt continuously using real-time feedback generated during active use.
• It employs techniques such as replay buffers, modular adaptation, and deferred decision processes to update policies and improve decision-making on the fly.
• Empirical evidence from robotics, language agents, and open-world recognition shows that LWD leads to significant improvements in success rates and operational efficiency.

Searching arXiv for papers on "Learning While Deploying" and closely related deployment-time learning paradigms. Learning While Deploying (LWD) denotes a family of deployment-time adaptation paradigms in which a model or decision system is not treated as fixed once it is in use. Across the literature, the phrase covers several closely related regimes: continual offline-to-online reinforcement learning for deployed robot fleets, online continual learning from non-resettable interaction streams, iterative deploy–collect–retrain–redeploy loops, retrieval- or memory-based post-deployment adaptation with frozen backbones, dynamic out-of-distribution detection with incremental class learning, and—in an earlier combinatorial-optimization formulation—adaptive control of decision granularity at inference time rather than a fixed one-element-per-step procedure (Wang et al., 1 May 2026, Fallah et al., 4 Jun 2026, Ye et al., 17 Mar 2026, Shuster et al., 2020, Wu et al., 11 Oct 2025, Kaymak et al., 20 Oct 2025, Ahn et al., 2020). The unifying theme is that deployment generates information—rollouts, interventions, interaction traces, measurements, failures, or execution memories—that is immediately or repeatedly reused to improve future deployed behavior.

1. Conceptual scope and problem formulation

A central motivation for LWD is that static offline training is misaligned with dynamic and open environments. Deployed systems encounter distribution shift, long-tail failures, new tasks, novel classes, sparse feedback, and human correction opportunities that are not fully covered by predeployment data (Wang et al., 1 May 2026). In language-agent settings, this mismatch is sharpened by the fact that agent actions and environmental transitions can only be sampled once per scenario and real-world environments cannot be trivially reset, so the learning problem is inherently one-shot and time-coupled to deployment (Fallah et al., 4 Jun 2026). In open-domain dialogue, the critique is directed at the “train once, test once” paradigm built on crowdsourced static datasets, which does not allow a model to learn from its experiences of using language (Shuster et al., 2020).

The literature does not use the term in a completely uniform way. In most recent work, LWD means post-deployment learning in the ordinary sense: the deployed system collects experience, updates a policy or auxiliary module, and redeploys the result (Wang et al., 1 May 2026, Fallah et al., 4 Jun 2026). In “Autonomous Learning after Model Deployment,” the same broad idea is formulated as continuous post-deployment detection of novel samples, labeling by human co-workers or other knowledgeable agents, and incremental class learning on the fly (Kaymak et al., 20 Oct 2025). In “Deployable Lifelong Learning,” the focus is on a pretrained system that, once deployed, continually recognizes, learns, and executes unseen tasks under constraints on memory, compute, and latency (Lekkala et al., 2023). By contrast, “Learning What to Defer” uses LwD to denote a deferred Markov decision process in which the agent learns how many element-wise decisions to make now and which ones to postpone, so deployment-time adaptation concerns stage decomposition rather than post-deployment parameter updates (Ahn et al., 2020).

This terminological spread suggests that LWD is best understood as a deployment-centered design principle rather than a single algorithm. The shared claim is that deployment is not merely inference. It is an information-generating phase that can alter subsequent behavior through online updates, replay, knowledge distillation, incremental statistics, decision rescheduling, or memory retrieval.

2. Core architectural patterns

Despite domain differences, LWD systems recur to a small set of architectural motifs. One is a closed loop of deployment, data collection, learning, and redeployment. The fleet-scale robot framework explicitly describes this as “deployment → shared physical experience → policy improvement → redeployment,” using autonomous rollouts and human interventions collected across a robot fleet and training from mixed replay over offline and online buffers (Wang et al., 1 May 2026). CLaaS exposes an analogous loop behind a chat API: a deployed agent produces a rollout $\tau_i$, the rollout and reward are inserted into an experience replay buffer $\mathcal{B}$, an asynchronous trainer samples minibatches from $\mathcal{B}$, the policy is updated, typically through LoRA adapter updates, and updated weights are hot-reloaded into the inference server (Fallah et al., 4 Jun 2026).

A second motif is reuse of scarce deployment data. Because deployment trajectories may be non-repeatable or expensive, replay is used to improve sample efficiency. CLaaS makes this explicit with a FIFO-style replay buffer controlled by $B_{\max}$, $B_{\min}$, and $A_{\max}$, and frames the key tradeoff as one between transfer and policy staleness (Fallah et al., 4 Jun 2026). OEL addresses the same scarcity constraint differently: raw user-side trajectories are converted into accumulated experiential knowledge $e_i$, and that knowledge is then consolidated into model parameters by on-policy context distillation on the server side, requiring no access to the user-side environment during training (Ye et al., 17 Mar 2026). The teacher is frozen, the student generates its own rollout response $y \sim \pi_\theta(\cdot \mid x)$, and optimization minimizes a reverse-KL objective conditioned on experiential knowledge.

A third motif is modularity between frozen pretrained components and lightweight online adaptation. PLDA keeps the feature extractor $f$ frozen, fixes a shared covariance matrix $\Sigma$, and updates only class means via

$\mathcal{B}$0

so post-deployment learning reduces to dynamic OOD detection plus incremental statistical updates rather than full retraining (Kaymak et al., 20 Oct 2025). Deployable Lifelong Learning similarly freezes a ResNet-based VAE encoder trained offline, updates only the last layer of a FSCIL task-mapper, and loads a stored task-specific 1-layer policy once the current task is recognized (Lekkala et al., 2023). Dejavu goes further by keeping the base Vision-Language-Action policy frozen throughout deployment and learning through an Experience Feedback Network (EFN) plus a growing memory bank of prior executions (Wu et al., 11 Oct 2025).

A fourth motif is direct use of deployment failures as supervision. LADs deploys generated cloud configurations in a Kubernetes-based multi-tenant environment, observes syntax failures, validator failures, runtime errors, warnings, benchmark outcomes, resource consumption, and cost-performance trade-offs, and feeds filtered logs and stage-specific failure points back into the next prompt through feedback-based prompt chaining (Khan et al., 28 Feb 2025). DISTINGUISH performs the same pattern in geosteering with a different mathematical substrate: real-time Logging-While-Drilling measurements are assimilated with an ensemble Kalman filter, and the updated geological ensemble is passed to a dynamic-programming-based decision support system for the next steering action (Alyaev et al., 11 Mar 2025).

3. Representative instantiations across domains

The range of LWD instantiations is broad enough that the term spans substantially different technical objects.

Setting	Deployment-time mechanism	Representative papers
Combinatorial optimization	Deferred element-wise decisions; adaptive number of stages	(Ahn et al., 2020)
Language agents and dialogue	Replay, parametric updates, experiential knowledge consolidation, redeployment from human interactions	(Fallah et al., 4 Jun 2026, Ye et al., 17 Mar 2026, Shuster et al., 2020)
Embodied and robotic control	Fleet-scale RL, human interventions, retrieved execution memories, residual correction	(Wang et al., 1 May 2026, Wu et al., 11 Oct 2025)
Open-world recognition and tasking	Dynamic OOD detection, incremental class means, FSCIL task mapping	(Kaymak et al., 20 Oct 2025, Lekkala et al., 2023)
Industrial control and infrastructure	EnKF-based state updating, dynamic programming, prompt-chained configuration repair	(Alyaev et al., 11 Mar 2025, Khan et al., 28 Feb 2025)

In combinatorial optimization, LwD was introduced to overcome the bottleneck that previous DRL solvers typically make one element decision per MDP step. For maximum independent set, the state is $\mathcal{B}$1, the action is a vector over deferred vertices $\mathcal{B}$2, and a clean-up phase restores feasibility. This allows the number of stages to shrink quickly on easy instances and stretch on hard ones, making the scheme more scalable on large locally-decomposable graph problems (Ahn et al., 2020).

In language settings, three deployment-time learning patterns appear. CLaaS performs asynchronous continual learning from a stream of scenarios through replay and on-policy policy-gradient updates such as PPO, REINFORCE++, and SDPO, with updated LoRA adapters hot-reloaded into service (Fallah et al., 4 Jun 2026). OEL extracts “transferable experiential knowledge” from user-side trajectories and then internalizes it by on-policy context distillation, with no need to revisit the environment during server-side training (Ye et al., 17 Mar 2026). The deployed fantasy dialogue game uses a repeated loop in which a retrieval model is deployed, human-model conversations are collected, all possible $\mathcal{B}$3 pairs are extracted, training uses only human utterances as targets, and the updated model is redeployed in subsequent rounds (Shuster et al., 2020).

In robotics, the fleet-scale framework starts from a pretrained VLA policy and combines Distributional Implicit Value Learning for robust value estimation with Q-learning via Adjoint Matching for policy extraction in flow-based VLA action generators (Wang et al., 1 May 2026). Dejavu instead treats post-deployment learning as retrieval-driven adaptation: a frozen VLA policy proposes a baseline action, EFN retrieves a semantically relevant prior experience from memory, predicts a residual correction $\mathcal{B}$4, executes the corrected action, compares the next observation with the retrieved successor observation, and inserts the new rollout into memory (Wu et al., 11 Oct 2025).

In open-world classification, ALMD requires the system to detect novel samples, obtain labels, and learn new classes immediately and incrementally. PLDA addresses this with LDA in frozen feature space, Mahalanobis- or Relative-Mahalanobis-based OOD scores, and continual expansion of the learned class set (Kaymak et al., 20 Oct 2025). DeLL treats deployment-time continual learning as task identification plus policy loading, with benchmark metrics including model size, inference time, learn switches, model growth, buffer size, and total normalized mean reward (Lekkala et al., 2023).

4. Empirical evidence

The empirical case for LWD is strongest where deployment-time feedback produces repeated quantitative improvement.

The fleet-scale robot study evaluates a single generalist policy on a fleet of 16 dual-arm robots across eight real-world manipulation tasks, including semantic grocery restocking and 3–5 minute long-horizon tasks, and reports that LWD (Online) reaches an average success rate of 95%, with the largest gains on long-horizon tasks (Wang et al., 1 May 2026). The same study reports a long-horizon average of 0.91 for LWD (Online), compared with 0.68 for SFT, 0.77 for RECAP, 0.73 for HG-DAgger, and 0.79 for LWD (Offline), together with a reduction in mean cycle time of 23.75 seconds.

CLaaS reports that parametric updates outperform in-context learning in an adversarial continual setting. In Table 1, ICL final performance is $\mathcal{B}$5, PPO final is $\mathcal{B}$6, REINFORCE++ final is $\mathcal{B}$7, and SDPO final is $\mathcal{B}$8; for continual-learning metrics, ICL forward is $\mathcal{B}$9 and forgetting is $\mathcal{B}$0, whereas SDPO forward is $\mathcal{B}$1 and forgetting is $\mathcal{B}$2 (Fallah et al., 4 Jun 2026). The paper summarizes this as parametric updates leading to superior forward transfer and less forgetting than in-context learning.

The deployed dialogue game reports progressive improvement across three rounds. Retrieval Hits@1/20 rises from 81.61 on the WILD test in Round 1 to 84.60 in Round 2 and 87.63 in Round 3, while generative perplexity on the WILD test falls from 13.42 to 12.31 to 11.79 (Shuster et al., 2020). Continue rate is reported in the range of 68%–75%, with the best listed value 75.2 ± 2.0% for the 90M Poly-Encoder trained on LIGHT+WILD with negative context training and decoding control.

The MIS study provides a distinct scalability result. On a 2-million-vertex BA graph, KaMIS achieves 909,988 in 3637 seconds, whereas LwD achieves 915,887 in 276 seconds, and the text characterizes this as about 13× faster (Ahn et al., 2020). The same paper states that S2V-DQN is not even feasible to evaluate on those million-scale graphs.

Post-deployment memory augmentation for embodied agents also yields large reported gains. On LIBERO, OpenVLA average success improves from 76.5 to 87.0 as memory volume rises to 1000, and UniVLA average success improves from 95.2 to 97.2 (Wu et al., 11 Oct 2025). On a real AgiBot G1 platform with the GO-1 policy, success improves from 46.9 to 68.8 on PutBottle, from 34.3 to 50.0 on SortItem, and from 15.6 to 43.8 on AddGoods.

In post-deployment class learning, PLDA with RMD is reported to achieve average F-score 78.67 and average accuracy 87.91, with post-deployment processing taking less than 15 ms (Kaymak et al., 20 Oct 2025). In DevOps, LADs reports that approximately 94% of configuration errors can be resolved through feedback-based prompt chaining by the third attempt (Khan et al., 28 Feb 2025).

5. Distinctions, misconceptions, and design trade-offs

One recurrent misconception is that LWD necessarily means online gradient updates of a single monolithic model. Several papers directly contradict that reading. Dejavu keeps the VLA backbone frozen and improves through retrieval, residual correction, and memory growth (Wu et al., 11 Oct 2025). PLDA performs no deep model fine-tuning during deployment, uses no replay buffer, and updates only class means in frozen feature space (Kaymak et al., 20 Oct 2025). DeLL likewise emphasizes lightweight online adaptation of a task-mapper rather than continual retraining of a shared control policy (Lekkala et al., 2023). Conversely, CLaaS and fleet-scale robot LWD do rely on ongoing parametric improvement and redeployment (Fallah et al., 4 Jun 2026, Wang et al., 1 May 2026).

A second misconception is that any deployment log is equally useful as supervision. OEL explicitly reports that extracted experiential knowledge is significantly more effective than raw trajectories, and that on-policy consistency between the knowledge source and the policy model is critical for effective learning (Ye et al., 17 Mar 2026). This suggests that representation and summarization of deployment experience can be as consequential as the volume of that experience.

A third issue concerns evaluation. Dialogue deployment demonstrates that automatic quality and human engagement need not coincide: the larger 622M model has worse continue rates than the 90M model even though it makes fewer factual and dialogue mistakes, with the paper attributing the difference to the larger model being perceived as more boring and repetitive (Shuster et al., 2020). The deployment-pipeline paper similarly argues against optimizing a single operational factor and instead evaluates learning cost for engineers, transparency, stability, security, parallelism, and cost (Li et al., 2022).

A fourth trade-off is economic rather than algorithmic. The learning-by-deploying industry model treats deployment as both output and data generation. Under exogenous prices, pooling raises welfare but firms underinvest in early deployment; under downstream Cournot competition, pooling depresses price and the private value of sharing can turn negative, with sustainability governed by demand elasticity over the output range induced by pooling (Tong et al., 30 Jun 2026). This indicates that deployment-time learning architectures interact with market structure, not only with model design.

6. Limitations and research directions

The current literature also identifies substantial open constraints. The fleet-scale robot framework states that its current online update schedule is simple, that long-horizon tasks use only one short language instruction, that safety is not explicitly modeled, and that actor–learner latency remains non-negligible, with episode-to-learner delay reported as P50 41 s and P99 148 s and model-published-to-actor delay as P50 38 s and P99 55 s (Wang et al., 1 May 2026). These are system-level limitations rather than isolated optimization details.

In geosteering, DISTINGUISH reports artifacts in GAN-generated geology, some “forgetting” because the EnKF does not reuse all previous data cumulatively in a way that fully resolves non-uniqueness, and residual uncertainty in stacked sand shapes even after 60 assimilated log locations (Alyaev et al., 11 Mar 2025). In cloud automation, LADs notes trade-offs among performance, cost, and scalability, and operational robustness depends on the quality of log analysis and repair prompting (Khan et al., 28 Feb 2025). In industrial deployment engineering, increasing automation can reduce transparency and complicate local debugging or Git coordination, even when it improves repeatability (Li et al., 2022).

Taken together, the literature presents LWD as a shift in the temporal boundary of learning. Training is no longer confined to a predeployment stage. Depending on the application, the deployed system may learn from autonomous rollouts and human interventions, one-shot scenario streams, human conversations, textual environment feedback, LWD measurements, runtime failures, retrieved execution memories, or dynamically expanding class sets (Wang et al., 1 May 2026, Fallah et al., 4 Jun 2026, Shuster et al., 2020, Ye et al., 17 Mar 2026, Alyaev et al., 11 Mar 2025, Wu et al., 11 Oct 2025, Kaymak et al., 20 Oct 2025). The strongest common claim is not that one mechanism dominates, but that deployment itself becomes a first-class source of optimization signal.