Agentic In-Context Learning

• Agentic in-context learning is a paradigm where LLMs are deployed as dynamic agents that plan, act, and update their memory based on structured feedback.
• It employs frameworks like POMDP, dynamic context pruning, and RL-driven skill induction to enable adaptive, autonomous tool use.
• Applications include reinforcement learning, multi-agent collaboration, and vision-language reasoning, demonstrating improved performance and efficiency.

Agentic in-context learning refers to the paradigm in which LLMs and related architectures are deployed not merely as static sequence predictors, but as agents that dynamically plan, act, observe, update memory, and adapt their behavior on the fly, using structured context and environmental feedback—without gradient-based parameter updates at inference time. This framework bridges classical in-context learning (ICL) with RL-driven agentic interaction and dynamic memory, creating systems capable of autonomous tool use, protocol orchestration, and intrinsic skill acquisition. Recent work formalizes and evaluates this paradigm across domains including reinforcement learning, skill induction, workflow automation, multi-agent collaboration, data science, and vision-language reasoning.

1. Formal Foundations: Agentic In-Context Learning as Agent-Environment Interaction

Classical ICL involves adapting to new tasks via examples placed directly in a static prompt, leaving model parameters fixed. Agentic in-context learning extends this model by allowing the agent to interleave reasoning, action selection (e.g., tool invocation), environment observation, context buffer updates, and reflection—instantiateable as a partially observable Markov decision process (POMDP) where the state at each timestep comprises input, context, tool outputs, and a history of actions, and the agent policy samples both internal thoughts and external actions (Wei et al., 18 Jan 2026).

Mathematically, an agentic in-context learner operates with a policy

$\pi_\theta(z_t, a_t \mid h_t)$

where $h_t$ encodes all available history (observations, thoughts, actions) up to timestep $t$, $z_t$ is the latent “thought,” and $a_t$ is the external action (including possible tool use or task completion signals). Decisions are made stepwise, with context windows or external memory holding sub-goal hypotheses, retrieved skills, or trajectory data. Rewards can be intrinsic (task success, compactness of internal context, etc.) or extrinsic (environmental feedback, validation accuracy) (Lu et al., 2 Apr 2026).

2. Agentic Skill Induction and Internalization

A fundamental challenge is to move beyond inference-time context augmentation—where agents simply read and follow skill packages from a retrieved memory—toward truly autonomous behavior in which skills are internalized into model parameters. The SKILL0 framework explicitly formulates this trajectory: skills, defined as structured bundles of procedural knowledge (Markdown-formatted files), are supplied in full context during training. Through a dynamic curriculum, skill context is progressively withdrawn according to on-policy helpfulness, measured via a metric

$$\Delta_k = \mathrm{Acc}_k^{\mathrm{w/skill}} - \mathrm{Acc}_k^{\mathrm{w/o \; skill}}$$

thus retaining only those skills from which the agent still benefits at each curriculum stage. The expected return incorporates both task reward and a compression incentive: $\tilde r_t = r_t + \lambda r_t^{\text{comp}}$ Training proceeds with PPO-style objectives and staged context pruning until the agent operates entirely zero-shot, with parameter-internalized skills: zero reliance on external skill retrieval at inference (Lu et al., 2 Apr 2026).

Empirically, on ALFWorld and Search-QA, SKILL0 showed +9.7% and +6.6% absolute gains, respectively, over RL baselines, while reducing average prompt length to <0.5k tokens/step. Ablation confirms the necessity of linearly decreasing skill budgets, three-step filter–rank–select curricula, and rapid context update intervals. This formalizes skill internalization as an agentic in-context learning target distinct from mere prompt engineering.

3. Evolution and Engineering of Agentic Context

Modern approaches recognize that not only skills but also context “engineering” must itself become agentic and adaptive. Meta Context Engineering (MCE) deploys a bi-level optimization: a meta-agent evolves context-engineering skills (i.e., how to construct, select, and adapt context), while a base-level agent optimizes the actual context artifacts (files, code, schemas) for each application instance (Ye et al., 29 Jan 2026). Skill evolution incorporates agentic operators, including crossover, mutation, and batch-level rollouts. The MCE framework outperforms alternative agentic context engineering methods (ACE, Dynamic Cheatsheet, etc.) by co-evolving context schemas and skillsets, yielding improvement ranges up to +53.8% on real-world tasks, more adaptable context sizes, and increased transferability.

ACE (Agentic Context Engineering) represents a modular evolution of context, composed of a generator to produce reasoning trajectories, a reflector to extract lessons learned, and a curator to incrementally update the context buffer in a manner that strictly avoids brevity bias and context collapse. Contexts thus become evolving, memory-rich playbooks rather than monolithic, statically-curated prompts, resulting in higher information retention rates and improved test accuracy across agent and domain-specific tasks (Zhang et al., 6 Oct 2025).

4. Substrate Modalities: Multi-Modal, Hierarchical, Experiential, and Segmented Agency

Agentic in-context learning extends beyond text-only domains, with systems such as ContextNav introducing a graph-based, agentic framework for multi-modal ICL: retrieval pipelines dynamically select, denoise, and structurally align multimodal (vision-language) context via feedback-driven graph orchestration (the Operational Grammar Graph), achieving robust performance against semantic and structural noise (Fu et al., 6 Oct 2025).

Segmentation-based approaches (e.g., JoA-ICL for news stance detection) instantiate agentic decomposition by delegating sub-tasks (such as short-segment stance labeling) to agents, with these segment outputs then aggregated via meta-prompts. This enables accurate inference in long-form contexts by exploiting hierarchical document structure, outperforming both standard ICL and full-article baselines on fine-grained classification tasks (Lee et al., 15 Jul 2025).

Experiential in-context learning benchmarks, such as BELA, probe multi-episode, adaptive behaviors: agents are evaluated not on single predictions, but on their capacity to accumulate, reason over, and leverage episodic memory throughout repeated interactions. Current models display limited improvement across episodes, indicating gaps in persistent memory, belief-update capability, and uncertainty calibration—areas prioritized in next-generation agentic architectures (Yang et al., 27 Nov 2025).

5. Workflow Instantiations: Algorithmic Patterns and Multi-Agent Orchestration

Agentic in-context learners are instantiated through principled workflow orchestration patterns, including sequential, modular, and collaborative processing:

• ReAct-style loops, where an agent alternates “thought” and “action” tokens, invoking tools, updating memory, and retrying sub-goals until task completion (Wei et al., 18 Jan 2026).
• Tree-of-Thought or MCTS-inspired search, where partial reasoning branches are evaluated and expanded adaptively, allowing for plan exploration and cost-controlled action selection.
• Nonlinear, multi-agent collaborative systems, exemplified by the AWE (Agentic Workflow for Education) framework, in which components for self-reflection, tool invocation, task decomposition, and agent communication are layered according to the von Neumann multi-agent system (MAS) architecture. This delivers flexible, robust, and personalized task completion in practical applications, such as domain-aligned exam item generation (Jiang et al., 1 Sep 2025).

Agentic in-context orchestration enforces context management via explicit memory curation, reflection-based self-improvement, function-based task decomposition, and context-driven agent-to-agent communication, delivering scalable and self-improving behaviors beyond what static demonstration-based prompting provides.

6. Applications, Empirical Findings, and Benchmarks

Agentic in-context learning has been applied in settings including multi-turn data science (CEDAR: orchestrator-agent plans and code generation with context-limited function interfaces) (Roy et al., 10 Jan 2026), code generation and review (ROME via ALE ecosystem: ROLL/ROCK/iFlow CLI pipeline with chunked RL and robust context engineering) (Wang et al., 31 Dec 2025), skill induction and dynamic demonstration selection for agentic task execution (Gupta et al., 16 Jun 2025), and tool-augmented reasoning with structured context management (GSM-Agent, which formalizes revisit, exploration, and exploitation in agentic reasoning graphs to analyze LLMs’ tool use and information collection) (Zhu et al., 26 Sep 2025).

Empirical findings demonstrate that agentic in-context learners can outperform static baselines in skill retention, memory fidelity, autonomous tool-use, and cross-task generalization, particularly when equipped with dynamic context pruning, curriculum-based skill withdrawal, modular memory buffers, and explicit action planning. Quantitative benchmarks show that chunked RL algorithms, dynamic context updates, and hierarchical agentic workflows are essential to robust, low-latency, zero-shot agentic competence.

7. Challenges and Future Directions

Major challenges include:

• Long-horizon credit assignment: Moving beyond heuristic value functions or simple task success signals, to develop more informative, self-generated feedback that supports multi-turn or multi-episode adaptation (Wei et al., 18 Jan 2026).
• Memory scalability and compression: Design of adaptive, context-aware retrieval and compression strategies to manage the growth of internal context/memory in long-horizon agentic tasks (Ye et al., 29 Jan 2026).
• World modeling and uncertainty quantification: Embedding probabilistic, neuro-symbolic belief update modules and persistent, structured memory into agentic inference loops, enabling reliable performance in open-ended and partially observed environments (Yang et al., 27 Nov 2025).
• Multi-agent and collaborative reasoning: Generalization of agentic orchestration to teams of role-specialized agents, with explicit protocols for communication, delegation, and artifact aggregation (Jiang et al., 1 Sep 2025).
• Governance and safety: Development of guard agents, constraint checking, and plan verification to ensure safe deployment of autonomous agents capable of self-directed tool use and environment modification (Wei et al., 18 Jan 2026).

Agentic in-context learning, by entwining dynamic context adaptation, internalization of procedures, and principled action planning within inference loops, defines a foundational direction for the evolution of highly autonomous, trustworthy AI agents across complex, real-world domains.