Retraining for In-Context Learning (RICL)

• Retraining for In-Context Learning (RICL) is a set of methodologies that modify training regimes to robustly induce few-shot learning in neural models.
• RICL employs techniques like reweighting algorithms, demonstration mining, and tailored pretraining to counteract prompt sensitivity and bias.
• RICL spans multiple domains, applying to language, vision-language, and real-time recommendation systems to enhance model adaptability and performance.

Retraining for In-Context Learning (RICL) is an umbrella term for methodologies designed to enhance the few-shot and interpretable in-context learning (ICL) capability of large-scale neural models—especially Transformers—by explicitly modifying the training regime, data, or downstream adaptation objective to induce or preserve ICL functionality. Unlike traditional pretraining or finetuning, RICL techniques operationalize in-context induction by leveraging weighted loss formulations, demonstration mining, tailored pretraining corpora, or architecture-level adjustments. The foundation of modern RICL lies in the realization that naive ICL is highly sensitive to prompt composition and is not always robustly induced by generic supervised or self-supervised pretraining alone.

1. Core Principles and Motivations

The central motivation for RICL arises from two empirical findings:

• Prompt sensitivity and bias: ICL performance is degraded by prompt imbalance, prompt noise, or subdomain shift, as standard models infer internal task variables (e.g., meta-parameters) from few-shot context via implicit optimization (softmax regression or least squares) (Chu et al., 2023).
• Emergent but unstable ICL: Pretrained Transformers can perform ICL incidentally on text, but these abilities are both fragile and highly dependent on properties of both the pretraining data and in-weight task structure (Bratulić et al., 9 Jan 2025).

RICL methods therefore aim to (a) simulate or reinforce in-context induction in the “training distribution” itself, (b) reweight context examples to counteract bias, or (c) modify architecture or downstream task protocols to enable robust ICL across modalities and application domains.

2. Reweighted In-Context Learning: Algorithmic Formalization

One canonical RICL methodology is the "Reweighted In-Context Learning" algorithm (Chu et al., 2023). Its design augments frozen LLMs with a lightweight reweighting schema over context examples:

• Problem formalism: Given a prompt $P = \{(x_1,y_1),\dots,(x_m,y_m),x_{query}\}$, the model’s output depends on an implicit parameter $x^*$. Prompt bias causes $x^*$ to drift from the optimal that would generalize to the data distribution.
• Reweighting objective: Introduce nonnegative weights $w=(w_1,...,w_m)$, defining

$$x^*(w) := \arg\min_{x\in\mathbb{R}^d} \sum_{i=1}^m w_i \cdot L(x;A_i,b_i),$$

where $L(x;A_i,b_i) = \frac{1}{2}\|f(A_i,x)-b_i\|^2_2$ measures context loss. The target is minimizing validation risk over a clean set $V$:

$$\mathcal{L}_{valid}(w) = \sum_{(A^v, b^v) \in V} \frac{1}{2}\|f(x^*(w);A^v) - b^v\|_2^2.$$

• Optimization (algorithm sketch): Freeze model parameters; iteratively optimize $w$ (and possibly an affine bias $B$ in the embedding layer) by backpropagating validation loss. This requires no update to underlying LLM weights.
• LARICL: For linear ICL, a closed-form $x^*$0 is derived, turning the problem into low-cost reweighting with convergence guarantees.

Empirical findings show RICL and its linear approximation LARICL outperform both naive ICL and full fine-tuning, preserving performance under context imbalance, high noise, and distribution drift.

3. Data-Driven RICL: Pretraining and Supportive Subset Selection

RICL can also refer to strategies that adjust the pretraining distribution to induce or amplify ICL capabilities. Two prominent directions are:

• Gradient-based data mining: Iteratively select pretraining data whose gradient aligns with the ICL objective gradient, producing a “supportive subset” $x^*$1. Continued pretraining on $x^*$2 can yield up to 18 percentage point improvement in downstream ICL accuracy without increasing zero-shot or standard task performance (Han et al., 2023). Properties of $x^*$3 include higher density of rare tokens, substantial long-range context, and no preferential domain relevance.
• Synthetic demonstration construction: Approaches like PICL construct training instances by retrieving and assembling paragraphs representing “intrinsic tasks” and learning to model them in an ICL format, which improves generalization compared to increasing model size alone (Gu et al., 2023).

The following table contrasts several RICL pretraining protocols:

Paper	Mechanism	Core intervention
(Han et al., 2023)	Gradient alignment	Continued PT on subset maximizing $x^*$4
(Gu et al., 2023)	Intrinsic task mining	Contrastive retrieval, ICL assembly, joint LM pretraining
(Bratulić et al., 9 Jan 2025)	Data curation	Explicit context repetition, increased task difficulty

4. Architectural and Modality Extensions

RICL is not restricted to LLMs. In vision-language-action (VLA) domains, RICL methods inject ICL adaptability into robotics foundation models (Sridhar et al., 4 Aug 2025):

• VLA RICL protocol: Post-train a pre-existing VLA (e.g., $x^*$5-FAST-DROID) to consume a retrieval buffer of $x^*$6 nearest-neighbor demonstrations in its context. The architecture is modified to include an “action-interpolation” layer that blends the retrieved demonstration with model outputs for robust in-context induction.
• Zero-shot transfer and adaptation: After RICL, the VLA can execute new tasks purely via demonstration retrieval in context, with no weight updates; further fine-tuning with task-specific data further boosts performance.

This paradigm achieves substantial improvements in real-world robotic task success rates in both online and offline learning environments, while maintaining low computational overhead and high data efficiency.

5. Practical Instances: Real-Time Recommendation and Streaming ICL

In online systems (e.g., recommendations), RICL is operationalized by tuning LLMs to always process real user feedback in an ICL prompt assembly, ensuring that native few-shot induction is preserved (Bao et al., 2024). Key properties:

• Prompt assembly: At each timestep, assemble the $x^*$7 most recent feedback-label pairs as context, train exclusively on such ICL-structured prompts. The forward (serving) API mirrors the training format.
• Benefits: Enables real-time personalization—models adapt instantly to unseen interests using demonstration-augmented prompts, without any parameter update.
• Empirical outcomes: AUC scores and update-gap (PDM) metrics show that RecICL-motivated RICL achieves or exceeds the gains of full retraining, with near-zero need for updates post-deployment.

A critical insight is that standard fine-tuning (“single-shot” examples only) destroys native ICL capabilities, while RecICL preserves them by always training with demonstration-based context.

6. Data Construction and Emergence of ICL via Retraining

Recent work demonstrates that the emergence and persistence of ICL circuits in autoregressive models are critically dependent on two training-time interventions (Bratulić et al., 9 Jan 2025):

• Exact repetition (iCopy) in context: Including even a single exact repetition of the query among demonstrations is sufficient to stabilize and amplify ICL in both vision and LLMs.
• Task hardness: Increasing class cardinality, label noise, or using instance-discrimination objectives induces non-transient ICL, even as the model remains trained via standard cross-entropy loss.
• Scheduling: Gradually reducing repetition (“burstiness”) during training preserves naturalness while maintaining ICL.

These effects are robust across modalities (vision, language, EEG), suggesting that retraining with context manipulation is a general recipe for ICL emergence.

7. Limitations, Open Problems, and Future Directions

Known limitations of current RICL techniques include:

• Validation set reliance: Methods like (Chu et al., 2023) require a small clean validation set to optimize reweighting, which may not exist in all application settings.
• Modality scope and theory: Most formal analysis applies to regression and classification; extension to complex generations or structured prediction is ongoing.
• Inference scaling: Increased context length and retrieval/assembly can raise inference latency, potentially necessitating further architectural innovations (e.g., efficient attention, prefill caching).
• Trade-offs in data curation: Emphasizing rare or challenging examples can harm base LM performance or increase exposure to artifacts, requiring careful mixing or curriculum designs (Han et al., 2023).

Emergent areas of RICL research entail: developing structured or parameter-efficient reweighting schemes, extending RICL-based protocols to broader sequence-to-sequence and chain-of-thought tasks, integrating collaborative and hierarchical feedback, and scaling modality-agnostic RICL to ever-larger and more diverse foundation models.

---

References:

• (Chu et al., 2023) Fine-tune LLMs to Approximate Unbiased In-context Learning
• (Han et al., 2023) Understanding In-Context Learning via Supportive Pretraining Data
• (Sridhar et al., 4 Aug 2025) RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Models
• (Bao et al., 2024) Real-Time Personalization for LLM-based Recommendation with Customized In-Context Learning
• (Gu et al., 2023) Pre-Training to Learn in Context
• (Bratulić et al., 9 Jan 2025) Unlocking In-Context Learning for Natural Datasets Beyond Language Modelling