Interactive In-Context Learning

• Interactive in-context learning is a dynamic paradigm that enables models to integrate real-time feedback during multi-turn interactions to continuously refine outputs.
• It leverages formal frameworks like POMDP formulations and information-maximization strategies to enhance learning across natural language, vision, and multimodal domains.
• Empirical results demonstrate significant performance gains and annotation efficiency, validating its potential for broad, cross-domain applications.

Interactive in-context learning (IICL) is a paradigm in which machine learning models improve their function dynamically over the course of multi-turn interactions, integrating real-time feedback, queries, or exemplars into their reasoning process. Unlike classical in-context learning, which operates over static, hand-crafted prompts or example sets, IICL is characterized by a closed feedback loop where a model’s outputs can elicit corrective guidance, clarifications, additional exemplars, or informative answers—enabling continual adaptation during dialog, task solving, or annotation. This paradigm encompasses natural language scenarios (teacher–student dialogues, feedback-driven reasoning, question-driven learning), active prompt engineering for LLMs, and extensions to multimodal and interactive computer vision tasks. IICL is now recognized as a distinct, trainable skill for large models, amenable to both reinforcement learning and supervised fine-tuning objectives.

1. Formal Frameworks and Taxonomy

IICL is formally instantiated as either a partially-observable Markov decision process (POMDP) or an interactive information-gathering protocol, depending on the domain and interaction style.

POMDP Formulation

In the RL^2F approach, the IICL problem is modeled as a POMDP $$\langle \mathcal{S}, \mathcal{A}, \mathcal{O}, T, R\rangle$$ where states $s_t = (k, o_t)$ aggregate conversation history $o_t$ with potentially privileged context $k$ (e.g., ground-truth labels or test outputs), actions $\mathcal{A}$ are natural-language utterances, and rewards are sparse, typically only granted at episode termination for correct solutions. The student and teacher policies $(\pi_S, \pi_T)$ alternate turns, with teacher interventions contingent on student correctness and always conditioned on hidden information $k$ (Klissarov et al., 17 Feb 2026).

Information Maximization (Question-driven Learning)

In INTERACT, the student LLM’s knowledge state $K_t$ evolves as $$c_{t+1} = c_t + \gamma g(c_t, q_t, a_t), \quad K_{t+1} = \phi(c_{t+1})$$ where questions $q_t$ and teacher responses $s_t = (k, o_t)$0 are dynamically selected to maximize expected knowledge gain $s_t = (k, o_t)$1. The goal is to maximize cumulative information gain or quiz accuracy given a budget of interaction rounds $s_t = (k, o_t)$2 (Kendapadi et al., 2024).

Multimodal and Computer Vision Domains

In CoDi-2 and WS-ICL, the protocol generalizes to cross-modal exemplar conditioning (text, image, audio, or weak visual prompts) and interleaved, multi-round session histories, with the model adjusting its output policy autoregressively based on all prior exchanges (Tang et al., 2023, Hu et al., 7 Oct 2025).

2. Representative Methodologies and Implementations

RL^2F: Reinforcement Learning with Language Feedback

RL^2F transforms any verifiable single-turn reasoning dataset into a didactic multi-turn interaction. The workflow is:

1. The student proposes a solution to a sampled problem.
2. If incorrect, the teacher provides a natural-language critique (privileged by access to the solution), shaping the next student response.
3. This loop unfolds up to $s_t = (k, o_t)$3 turns with only the final correct answer rewarded. Training employs policy gradients (PPO) and an auxiliary supervised loss for teacher utterance prediction:

$s_t = (k, o_t)$4

This “imitation” term supports the model’s eventual capacity for self-generated feedback, closing the loop on autodidactic learning (Klissarov et al., 17 Feb 2026).

INTERACT: Student-Teacher Question-Driven Learning

A student LLM, initially with minimal knowledge, iteratively probes a teacher model, which answers based on privileged access to a knowledge source (document D, image). The student updates its internal state after each answer, optionally summarizing the dialogue. The formal learning objective is to maximize cumulative information gain—effectively, to design or select questions that efficiently close the knowledge gap relative to an upper bound (teacher). Implementation is agnostic to LLM architecture and is validated across 1,347 diverse real-world documents, images, and scenarios (Kendapadi et al., 2024).

ScatterShot: Interactive Example Curation

ScatterShot augments classical few-shot prompt engineering via an active, slice-based interaction loop. It clusters unlabeled data by key-phrase patterns, identifies underexplored “slices,” and actively solicits user evaluations or corrections for model outputs. A UCB-style reward score prioritizes candidate examples, maximizing exploration of error-prone or rare slices. An annotation UI accelerates curation with LLM-assisted pseudolabeling once model agreement is high, minimizing annotation effort and improving coverage of edge-case functions (Wu et al., 2023).

Multimodal Interactive ICL

CoDi-2 exemplifies a fully interleaved, token-stream paradigm in which text, images, and audio are embedded as contiguous tokens/features for both input and output. The model’s attention window naturally supports in-context learning from mixed-modality exemplars, instructions, and edits across arbitrary turn sequences. Training employs combined autoregressive language modeling, feature alignment, and diffusion-based reconstruction losses, enabling robust zero-shot and interactive generalization (Tang et al., 2023).

Medical Vision: Weakly-Supervised In-Context Learning (WS-ICL)

WS-ICL defines an efficient “prompt-annotated” conditioning mechanism, where weak prompts (2D boxes, sparse points) in the context set replace dense masks for segmentation. A dual-branch U-Net fuses target scan streams with context-branch averages, allowing strong in-context adaptation at much lower annotation cost. Annotation-efficient protocols reduce labeling time by over 96% without significant loss of performance (within 2.5 Dice points of full supervision) (Hu et al., 7 Oct 2025).

3. Empirical Findings and Benchmarks

Performance Impact

• RL^2F-trained models exhibit dramatic multi-turn performance improvements: e.g., Gemma 3 12B with RL^2F gains ~15 percentage points on Omni-MATH by turn 5; outperforms single-turn RL on cross-domain transfer by +8 points (LiveCodeBench), reaching parity with models an order of magnitude larger (Klissarov et al., 17 Feb 2026).
• INTERACT yields up to 25% absolute improvement in quiz accuracy (gpt-4o-mini: 47.9% to 73.7% over 5 turns), enabling cold-start students to reach static-lesson baselines within five queries. Benefits are robust to teacher model strength and cannot be replicated via passive transcript “replay” (Kendapadi et al., 2024).
• ScatterShot’s slice-based active example sampling outperforms random curation by 4–5 points on text transformation benchmarks and halves variance in few-shot prompt effectiveness (Wu et al., 2023).
• CoDi-2 surpasses previous multimodal models on subject-driven image generation (CLIPSim 0.703), audio editing, and multi-turn compositionality (Tang et al., 2023).
• WS-ICL achieves within 2.5 Dice points of fully supervised 3D segmentation with 3% of the annotation time; in some tasks, weak prompt conditioning is nearly optimal (Hu et al., 7 Oct 2025).

Plasticity and Self-Improvement

Explicit training for IICL results in pronounced in-context plasticity: RL^2F models prove capable of rapidly reorganizing reasoning or solution strategies in response to feedback, a property not observed in static or “acknowledgment-only” baselines. Ablations confirm that multi-turn exposure and auxiliary critique losses are critical; removal of either collapses performance gains or self-improvement (Klissarov et al., 17 Feb 2026).

4. Domain and Task Generalization

IICL demonstrates compelling cross-domain robustness:

• RL^2F-trained models show out-of-distribution generalization from math to coding, puzzles, and navigation, with multi-turn gains persisting in held-out domains (Klissarov et al., 17 Feb 2026).
• INTERACT is validated across 1,347 textual and visual domains, including music lyrics, news, academic abstracts, and VQA images, underscoring the protocol’s applicability outside of “scripted” NLP (Kendapadi et al., 2024).
• Multimodal IICL (CoDi-2) supports true any-to-any cross-modal synthesis: inference histories include not only exemplars but also newly generated, edited, or composed samples—demonstrating transfer and few-shot adaptability in non-text tasks (Tang et al., 2023).
• In medical vision, WS-ICL generalizes reliably to new imaging modalities and anatomies, including challenging benchmarks (FLARE22, NasalSeg, Mice μCT) (Hu et al., 7 Oct 2025).

5. Practical Architectures and Optimization Strategies

Approach	Domain(s)	Model Core
RL^2F	Text reasoning	Transformer (Gemini, Gemma)
INTERACT	Text, vision	Open LLMs (GPT-4o, LLaMA)
ScatterShot	Text transformation	GPT-3, prompt curation UI
CoDi-2	Multimodal	Llama-based, ImageBind
WS-ICL	Medical imaging	Dual-branch U-Net

• RL^2F, INTERACT, and ScatterShot are compatible with, and benefit from, public LLM architectures equipped with transformer attention. RL^2F and INTERACT specifically demonstrate improvements as a function of model size, with smaller models closing the performance gap via multi-turn training.
• CoDi-2 illustrates that mixed-modality tokenization and continuous feature handling via diffusion decoders are tractable within autoregressive transformer backbones.
• WS-ICL shows that a dual-branch multi-scale encoder with prompt-fusion generalizes the IICL concept to spatial vision problems.

6. Cost, Annotation, and Human Interactivity

Annotation cost, user burden, and human–AI synergy are central in IICL research:

• ScatterShot improves annotation efficiency by pairing model-auto-generated suggestions with slice-based sampling; pseudo-labeling and batch evaluation further reduce necessary human corrections (Wu et al., 2023).
• WS-ICL documents a >30x annotation time reduction when using sparse box prompts relative to full 3D segmentation masks, almost saturating model performance curves (Hu et al., 7 Oct 2025).
• INTERACT highlights the superiority of active, personalized questioning versus passive observation or static summaries, both in speed to mastery and in closing capability gaps (Kendapadi et al., 2024).

7. Limitations, Open Challenges, and Future Directions

Limitations and open questions identified include:

• Dependence on context window size—especially acute in multimodal scenarios (e.g., CoDi-2)—limits session length and historical recall (Tang et al., 2023).
• RL^2F and related methods are sensitive to the structure and richness of feedback; poorly designed teacher signals can hinder learning or encourage sycophancy (Klissarov et al., 17 Feb 2026).
• ScatterShot’s pattern clustering is reliant on key-phrase extraction and may not readily translate to highly abstractive or translation tasks; adaptation to more general blind-spot discovery is proposed (Wu et al., 2023).
• WS-ICL is optimal for tasks with well-defined, high-contrast boundaries; for ambiguous anatomical targets, dense annotation may still be preferable as a second pass (Hu et al., 7 Oct 2025).
• Future research targets include curriculum design for problem selection, mechanisms for memory consolidation of in-context updates (continual learning), robust handling of adversarial or manipulative feedback in debate or negotiation, and tighter integration of user feedback loops for real-time model refinement.

Interactive in-context learning is established as a general, scalable, and methodology-rich paradigm for dynamic adaptation, cross-domain generalization, annotation efficiency, and model–user co-adaptation across natural language, vision, and multimodal ML systems (Klissarov et al., 17 Feb 2026, Kendapadi et al., 2024, Tang et al., 2023, Wu et al., 2023, Hu et al., 7 Oct 2025).