Learning from Inference-Time Execution (LITEN)

• LITEN is a paradigm that transforms inference-time computational artifacts into structured, persistent memory modules without altering model weights.
• It employs modular soft prompts and reinforcement learning to extract and inject execution insights for improved future decision-making.
• Empirical results show that LITEN achieves notable gains in diverse domains like mathematical reasoning, robotics, and GUI automation.

Learning from Inference-Time Execution (LITEN) is a paradigm that enables AI agents—particularly LLMs and multimodal systems—to harvest, structure, and reuse knowledge gleaned from their own inference-time computations and explorations. Instead of remaining stateless, LITEN-equipped systems systematically convert ephemeral execution traces produced during test-time into persistent, actionable knowledge modules, enabling on-the-fly improvement and adaptation without parameter updates or external labels. The LITEN framework draws upon recent research on memory-augmented reasoning, unsupervised reinforcement learning, and efficient (non-parametric) continual adaptation across a broad spectrum of domains, including language modeling, mathematical and logical reasoning, robotics, time-series analysis, and GUI automation.

1. Core Principles and Formal Definitions

LITEN is defined by its capacity to convert inference-time computational artifacts—reasoning traces, trajectories, tool-use executions, or exploration branches—into persistent, structured representations that can be retrieved and injected as context for future problem instances. The base model’s parameters remain unaltered; all adaptation arises from dynamically constructed external modules or memory, often attached as prompt augmentations, soft prompts, or in-context demonstrations.

Mathematically, for a pre-trained model $M_{\text{base}}$ solving query $x$, execution produces a trace $E(x)$ (e.g., chain-of-thought, subtask sequence, action trajectory) along with internal artifacts such as hidden states or subgoal decomposition. LITEN mechanisms extract, summarize, and structure such artifacts into a memory $\mathcal{M} = \{m^{(i)}\}$, external to $M_{\text{base}}$. Future queries retrieve $m^*$ via a retrieval function, e.g., $$m^* = \arg\max_{m \in \mathcal{M}}\cos(\text{Embed}(x), k_m)$$, and augment $M_{\text{base}}$'s input accordingly (Dorovatas et al., 16 Jun 2026).

Theoretical underpinnings show that, under unbounded resources, any capability learned via supervised fine-tuning can in principle be replicated by dynamically assembling sufficient inference-time context (in-context learning), establishing that LITEN subsumes parametric fine-tuning in capability—though efficiency and generalization are strongly dependent on context window limits and memory organization (Sharma, 9 Jun 2025).

2. Memory Structures, Storage, and Retrieval

LITEN instantiates external persistent knowledge via modular, compact memory structures. In ELM (Experiential Latent Memories), each experience is encoded as a lightweight soft prompt—a set of $k$ trainable vectors in $\mathbb{R}^d$ requiring $x$0 of model parameters—which never alter core weights and can be trained in a handful of gradient steps per instance. Memories are strictly modular: each corresponds to a single encountered sample, enabling fine-grained addition and removal without catastrophic forgetting.

Retrieval involves embedding new queries and searching for similar memory keys (mean-pooled prompt activations or learned key vectors). Upon retrieval, the most relevant memory is prepended as a soft prompt for inference. To prevent memory conflicts and regression, a lightweight verifier arbitrates between zero-shot and memory-augmented predictions (Dorovatas et al., 16 Jun 2026).

Hierarchical and structured memory is also employed, especially in domains requiring multilevel experience (e.g., TimeClaw for time-series analysis). Here, memory modules may include "Notes" (raw append-only evidence), structured rules, tool-usage notes, and explicit skill procedures, each annotated with confidence scores and conditions for application. Rules transition from passive to inject-active status based on accumulated confidence (Liu et al., 11 May 2026).

3. Learning from Inference-Time Signals

Central to LITEN is learning directly from the results of inference-time computation, using internally generated or easily verifiable signals. Several methods provide distinct instantiations:

• Majority Voting Rewards: In ELM, extra inference-time rollouts yield a distribution of answers per query $x$1; the mode serves as a noisy pseudo-label. Individual soft prompts are updated with a simple policy gradient, optimizing the loss

$x$2

where $x$3 if $x$4 and $x$5 otherwise. Training uses Group Relative Policy Optimization (GRPO) for efficient gradient signal (Dorovatas et al., 16 Jun 2026).

• Trajectory-Level Reinforcement Learning: Systems such as InftyThink+ cast iterative chain-of-thought as an episodic RL problem, assigning rewards based on final answer correctness and efficiency of the reasoning trajectory:

$x$6

and updating policies with trajectory-level GRPO (Yan et al., 6 Feb 2026).

• Metric-Supervised Distillation: TimeClaw leverages ground-truth numeric metrics to compare multiple tool-augmented execution branches, selecting the best-performing trajectory for experience distillation (Liu et al., 11 May 2026).

Learning is local (per-sample or per-scope), model weights are not updated online, and unreliable or noisy signals are filtered via structured assessment or verifier modules.

4. Domain-Specific Extensions and Applications

LITEN has been demonstrated across a spectrum of domains with specialized adaptations:

• Mathematical and Symbolic Reasoning: ELM, InftyThink+, and similar frameworks outperform both zero-shot and in-context learning baselines on challenging benchmarks (e.g., MATH500, AMC23, AIME24) and achieve gains comparable to full fine-tuning, but with frozen model parameters and much smaller resource footprints (Dorovatas et al., 16 Jun 2026, Yan et al., 6 Feb 2026).
• Vision-Language-Action (VLA) Models and Robotics: In robotic control, LITEN enables a high-level vision-language planner to incorporate execution outcomes (success/failure, outcome descriptions, causal reasoning) from repeated low-level policy trials into its in-context planning. Iterative structured assessment, rather than gradient updates, drives rapid improvement in success rates across diverse manipulation tasks (Shah et al., 22 Oct 2025).
• Computer-Use Agents and GUI Automation: Agents equipped with LITEN can harvest demonstration trajectories from external video tutorials—via segmentation, structured action extraction, and hierarchical in-context selection—yielding higher automation success on desktop and web tasks than agents using only text or static transcripts (Liu et al., 6 Nov 2025).
• Time-Series Analysis: In TimeClaw, agent executions on forecasting/monitoring tasks are compared using domain-relevant numeric metrics, and distilled procedural knowledge (including tool-choice strategies) is injected non-parametrically for future queries. Task-aware tool dropout prevents "tool-prior collapse," promoting exploration and better skill acquisition (Liu et al., 11 May 2026).

5. Empirical Results and Ablation Evidence

LITEN-based systems consistently yield significant empirical gains over both parameter-frozen in-context learning and more traditional training-based baselines. Key results (greedy pass@1 accuracy, success rates):

System/Benchmark	Baseline	LITEN Variant	Absolute Gain
ELM (MATH500, LLaMA)	45% (Zero-shot)	51.4% (ELM)	+6.4 pts
ELM (AMC23, LLaMA)	20%	26%	+6 pts
InftyThink+ (AIME24)	29.5%	50.9%	+21.5 pts
TimeClaw (MTBench)	Baseline Guillotine	+ (Consistent gains)	+ (Task-dependent)
Robotics (Stacking)	<20% (No-Feedback)	60–80% (LITEN)	+40–60 pts
GUI (OSWorld)	46.8% (Base)	50.3% (Video Demo)	+3.5 pts

Ablation studies show that: (i) modular soft-prompt memory outperforms LoRA and Prefix-Tuning for both offline and continual-augmentation tasks; (ii) memory consolidation, key/value selection, and rapid gradient updates enable gains with minimal compute; (iii) structured negative feedback (failures) is essential—restricting in-context memory to only successes underperforms; (iv) hierarchical segmentation and dynamic retrieval in video-based LITEN is necessary for transfer beyond static text guides (Dorovatas et al., 16 Jun 2026, Shah et al., 22 Oct 2025, Liu et al., 6 Nov 2025).

6. Comparison with Related Paradigms and Limitations

LITEN subsumes and extends in-context learning, retrieval-augmented generation (RAG), and self-reflection (e.g., Reflexion, ReAct). Unlike classic ICL or RAG, which supply frozen example batches or database lookups, LITEN (1) adapts memory content based on recent execution traces, (2) admits non-parametric continual accumulation, and (3) leverages structured internal signals (metrics, verifier judgments) as internal reward.

A key distinction is that LITEN decouples learning and parametric change: persistent improvement is achieved at the memory/prompt level, not by overwriting or fine-tuning model weights, thereby avoiding catastrophic forgetting and supporting compositional adaptation. Tradeoffs include increased memory bank management and retrieval complexity, and (in soft-prompt approaches) potential prompt-window saturation in long-running deployments (Sharma, 9 Jun 2025, Dorovatas et al., 16 Jun 2026).

Limitations identified include: (i) reliance on internal or proxy rewards can propagate error if uncalibrated; (ii) context window or prompt length can bottleneck where compressive or hierarchical summarization is not employed; (iii) in physical or non-symbolic domains, causal reasoning across subtasks remains a challenge for current VLM architectures; (iv) some approaches (e.g., positive-only ICL) fail to leverage negative feedback, leading to suboptimal performance (Shah et al., 22 Oct 2025).

7. Directions for Extension and Open Challenges

Methods for experience consolidation (merging, pruning, and organizing memory modules), advanced retrieval (richer embedding spaces, key-value learnability, hierarchical selectors), and dynamic adaptation (verifier calibration, adaptive gradient step scheduling) represent promising directions for improving memory efficiency and scaling.

For robotics and time-series regimes, more powerful causal reasoning and cross-subtask logic remain open challenges. Automated curriculum design, integration of richer reward modeling, and joint parametric–nonparametric learning are listed as future directions for broadening LITEN’s applicability to open-domain settings and lifelong learning scenarios (Shah et al., 22 Oct 2025, Liu et al., 11 May 2026).

A plausible implication is that as model architectures and task complexities scale, LITEN-style nonparametric self-improvement will form a core component in any efficient, robust, and general continual learning ecosystem for both language and multimodal agents.