Test-Time Textual Learning Overview

• Test-time Textual Learning (TTL) is a methodology where models update parameters or prompts using deployment-specific text, enabling dynamic adaptation.
• TTL techniques use gradient-based updates, self-supervised strategies, retrieval-fine tuning, and prompt rewriting to address distribution shifts and improve output quality.
• Practical implementations of TTL show gains in reading comprehension, language modeling, and vision-language tasks, albeit with increased computational costs.

Test-time Textual Learning (TTL) denotes a family of inference-time adaptation methods in which a model improves on a target instance, stream, or task by learning from text available only at deployment. In the recent literature, the phrase covers several distinct mechanisms: gradient-based test-time training on in-context demonstrations, self-supervised adaptation on unlabeled test text, retrieval-conditioned fine-tuning on nearest-neighbor text, pseudo-label- and reward-driven reinforcement updates, cross-episode prompt rewriting for language agents, and fully textual optimization procedures that do not update weights. A common distinction is that ordinary in-context learning keeps parameters fixed, whereas many TTL methods update parameters or prompts using test-time evidence; however, some works treat prompt rewriting or textual skill synthesis as TTL even when the actor weights remain frozen (Gozeten et al., 14 Mar 2025, Hu et al., 27 May 2025, Lou et al., 1 Apr 2026, Ye et al., 17 Apr 2026).

1. Terminology, lineage, and scope

An early explicit precursor appears in unsupervised reading comprehension, where a model adapts separately to each test passage by generating synthetic question-answer pairs from that passage and then answering human-authored questions about the same context. That work treats each new context as its own distribution and describes the procedure as “distribution-blind,” because the relevant adaptation signal is the passage itself rather than a global supervised corpus (Banerjee et al., 2021).

Later work broadened the setting from passage-specific extractive QA to general language modeling and LLMs. "Test-Time Training on Nearest Neighbors for LLMs" introduced a baseline in which the model retrieves similar text from a large index and fine-tunes on those neighbors immediately before predicting the test example, then resets parameters after the example is finished (Hardt et al., 2023). In 2025, TTL terminology became more explicit in LLM adaptation papers: "Test-Time Learning for LLMs" defines TTL as adaptation during deployment using only unlabeled test inputs from a shifted target distribution, while "Test-Time Training Provably Improves Transformers as In-context Learners" analyzes a setting where in-context demonstrations in the prompt are reused as a miniature supervised training set at inference time (Hu et al., 27 May 2025, Gozeten et al., 14 Mar 2025).

Subsequent work widened the phrase further. Some papers use TTL for cross-episode improvement in language agents, where a frozen actor is updated only through prompt rewriting by a meta-agent, formalized by

$\pi_{k+1} = f(\pi_k, \mathcal{H}_k).$

Others use TTL for test-time reinforcement learning, test-time curriculum synthesis, or even text-side prompt learning in vision-language OOD detection (Lou et al., 1 Apr 2026, Yang et al., 30 Jan 2026, Ye et al., 17 Apr 2026). This suggests that TTL is best understood as an umbrella term for inference-time learning from textual or text-mediated evidence, rather than a single algorithmic recipe.

2. Principal adaptation regimes

Recent TTL methods differ primarily in what is updated, what supervision is available at test time, and whether the adaptation signal comes from the prompt itself, unlabeled test text, retrieved data, or repeated interaction.

Regime	Core mechanism	Representative paper
Prompt-conditioned parameter update	One gradient step on in-context demonstrations before prediction	(Gozeten et al., 14 Mar 2025)
Unlabeled test-text adaptation	Input perplexity minimization with Sample Efficient Learning Strategy and LoRA	(Hu et al., 27 May 2025)
Retrieval-conditioned fine-tuning	Fine-tune on nearest-neighbor text, then reset	(Hardt et al., 2023)
Passage-specific self-supervision	Generate synthetic QA pairs from the test passage itself	(Banerjee et al., 2021)
RL-style self-improvement	Pseudo-labels or verifier rewards drive GRPO-style updates	(Liu et al., 15 Aug 2025)
Weight-free cross-episode adaptation	Meta-agent rewrites actor prompt across episodes	(Lou et al., 1 Apr 2026)
Fully textual optimization	Textual self-attention and textual gradients without parameter updates	(Mo et al., 10 Nov 2025)

The contrast between standard in-context learning and TTL is especially sharp in the transformer-theory literature. In the analyzed TTT setting, the prompt contains labeled demonstrations $(x_i,y_i)$ and a query $x$, with tokens represented as $z_i=[x_i;y_i]$ and $z=[x;0]$. A pretrained sequence model is then updated on a test-time set $S_{TT}$ by one gradient step:

$$W_{TT} := W^\ast - \eta \nabla \widehat L_{S_{TT}}(W^\ast).$$

The central claim is that the prompt is no longer only context; it is also a training set (Gozeten et al., 14 Mar 2025).

By contrast, TLM for LLMs uses only unlabeled test text and formulates adaptation as input perplexity minimization. For a token sequence $\{x_1,\dots,x_T\}$, perplexity is

$$\mathcal{P}(\{x_1,x_2,\dots,x_T\}) = e^{\left(-\frac{1}{T}\sum_{t=1}^{T}\log p(x_t \mid x_{1:t-1};\Theta)\right)}.$$

The method updates LoRA parameters to reduce input perplexity, emphasizes high-perplexity samples through a Sample Efficient Learning Strategy, and uses LoRA rather than full-parameter optimization to mitigate catastrophic forgetting (Hu et al., 27 May 2025).

A different axis of variation concerns whether learning occurs in parameter space or text space. Meta-TTL, SkillTTA, and TSAN keep the actor or solver fixed and adapt through prompt rewriting, temporary skill synthesis, or iterative textual attention and critique. These methods are presented as TTL because behavior changes at test time through structured textual updates, even though weights need not change (Lou et al., 1 Apr 2026, Wang et al., 16 May 2026, Mo et al., 10 Nov 2025).

3. Theoretical accounts: alignment, distribution shift, and context efficiency

The most explicit theoretical account of TTL in transformers appears in the linear-transformer analysis of test-time training for in-context learning. For a one-layer linear transformer / linear attention model,

$SM(Z,W)=x^\top W X^\top y,$

Proposition 3.1 shows that one gradient step induces a rank-one update

$(x_i,y_i)$0

The paper argues that this makes adaptation at test time “extremely structured”: it is driven by residuals on the test-time examples and is low-rank (Gozeten et al., 14 Mar 2025).

In the isotropic setting $(x_i,y_i)$1 with noiseless labels, the pretrained optimum satisfies

$(x_i,y_i)$2

The optimal step size and single-step improvement scale as

$(x_i,y_i)$3

The interpretation given is that improvement grows with the number of test-time training samples $(x_i,y_i)$4, depends strongly on context length $(x_i,y_i)$5, and can reduce the context needed for successful in-context learning. Under isotropic priors, standard ICL needs about $(x_i,y_i)$6 context length, whereas TTT can succeed with $(x_i,y_i)$7 context length by effectively memorizing the task from test-time examples (Gozeten et al., 14 Mar 2025).

The same paper isolates the role of alignment under general covariances through

$(x_i,y_i)$8

where $(x_i,y_i)$9 is the misalignment term and $x$0 captures signal power. With optimal step size,

$x$1

The paper’s interpretation is that when the pretrained model is misaligned with the target task, $x$2 is larger and TTT yields larger improvement, so TTT can partially undo distribution shift (Gozeten et al., 14 Mar 2025).

A nonlinear extension appears in "Test time training enhances in-context learning of nonlinear functions." There the task family is a single-index model

$x$3

with $x$4 drawn from a hidden $x$5-dimensional subspace. The paper analyzes a single-layer transformer with staged TTT: weak recovery via self-distillation, strong recovery via online SGD on true labels, and final MLP adaptation to the nonlinear link. Its main bound states that the prediction risk can approach the noise floor:

$x$6

The central claim is that TTT allows adaptation to both the feature vector $x$7 and the task-varying link function $x$8, whereas ICL alone is theoretically limited in adapting to shifts in the link function (Kuwataka et al., 30 Sep 2025).

A related but architecturally different theoretical framing appears in long-context language modeling. "End-to-End Test-Time Training for Long Context" formulates long-context LM as continual learning rather than architecture design: the model processes a sequence and keeps updating MLP weights via next-token prediction,

$x$9

with a training objective that mirrors the post-adaptation test-time loss. This work argues that test-time learning can compress context into weights and attain full-attention-like scaling with context length while retaining constant inference latency (Tandon et al., 29 Dec 2025).

4. Self-supervision, retrieval, and unlabeled-text adaptation

A major branch of TTL uses self-supervision from text that is present only at test time. In reading comprehension, the model receives a single passage $z_i=[x_i;y_i]$0, generates $z_i=[x_i;y_i]$1 synthetic QA pairs $z_i=[x_i;y_i]$2, trains on those pairs with extractive answer-span loss, and then answers unseen human-authored questions about that same passage. The paper studies single-context TTL, $z_i=[x_i;y_i]$3-neighbor test-time RC, curriculum test-time RC, and online test-time RC; its best online $z_i=[x_i;y_i]$4-neighbor variant reaches 69.8 EM / 80.4 F1 on SQuAD 1.1 for BERT-Large TTL* and 38.2 EM / 52.6 F1 on NewsQA for BERT-Large TTL* (Banerjee et al., 2021).

Retrieval-based TTL for language modeling takes a different route. TTT-NN retrieves nearest neighbors from a large distributed index over the Pile training set, fine-tunes the LM sequentially on those neighbors, evaluates on the test instance, and then resets parameters. The main setting uses 50 nearest neighbors, but most of the gain already appears after 20 neighbors. The paper reports that training on just 50 neighbors reduces bits per byte by about 20% on average, with reductions of more than 60% on pile_github, and that TTT-NN can narrow the performance gap between a 117M GPT-2 Small and a GPT-Neo model more than 10 times larger (Hardt et al., 2023).

TLM generalizes self-supervised TTL to unlabeled deployment streams for LLMs. Its central empirical and theoretical claim is that minimizing input perplexity can improve output perplexity. The paper reports that in 400 batches from DomainBench, 98.75% of batch samples satisfied the non-negativity condition for the gradient inner product, with average inner product $z_i=[x_i;y_i]$5. It introduces AdaptEval, containing DomainBench, InstructionBench, and ReasoningBench, and evaluates Llama3.2-3B-Instruct, Llama3-8B-Instruct, Llama2-13B-chat, and Qwen2.5-7B-Instruct. Reported gains include at least 20% improvement on domain knowledge adaptation, with Geography on Llama3.2-3B-Instruct improving from 0.2395 to 0.2893, Alpaca-GPT4 on Llama3-8B-Instruct improving from 0.3752 to 0.4274, and GSM8K improving from 0.7610 to 0.8074 (Hu et al., 27 May 2025).

These self-supervised lines share a common operational theme: the test passage, the unlabeled test stream, or retrieved neighbor text is converted into a temporary adaptation set. A plausible implication is that TTL replaces part of static pretraining with local specialization under the actual deployment distribution.

5. Reinforcement learning, curricula, and cross-episode adaptation

Another major branch of TTL uses pseudo-labels, self-consistency, or verifiable rewards. In ETTRL, test-time reinforcement learning samples multiple responses, uses majority voting over extracted answers to define pseudo-labels, and updates the model with GRPO-like optimization. The paper identifies two failure modes in prior TTRL—high inference cost and early-stage estimation bias—and addresses them with Entropy-fork Tree Majority Rollout (ETMR) and Entropy-based Advantage Reshaping (EAR). Under $z_i=[x_i;y_i]$6, ETMR uses about 60% of the token cost of fully parallel sampling, and the paper reports a 68 percent relative improvement in Pass at 1 on AIME 2024 for Llama3.1-8B while consuming only 60 percent of the rollout tokens budget (Liu et al., 15 Aug 2025).

TTCS targets difficult reasoning problems where raw test questions are too hard to yield reliable pseudo-labels and small test sets make online updates unstable. It initializes a question synthesizer and a reasoning solver from the same pretrained model, trains both with GRPO, and rewards synthetic questions near the solver’s capability frontier through

$z_i=[x_i;y_i]$7

The paper reports that TTCS raises the average score of Qwen2.5-Math-1.5B from 17.30 to 41.49, reaches 52.54 for Qwen2.5-Math-7B, and transfers gains to MMLU-Pro and SuperGPQA (Yang et al., 30 Jan 2026).

"Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning" combines automatic curriculum construction via SIFT with GRPO updates on a task-specific subset of a large verifiable corpus. For Qwen3-8B, TTC-RL improves AIME25 pass@1 from 23.33 to 41.67, CodeElo pass@1 from 13.73 to 29.34, and raises AIME25 pass@8 from 40% to 62% (Hübotter et al., 6 Oct 2025). In a related agentic direction, Meta-TTL treats the adaptation policy itself as the object of learning. The actor policy is frozen, the meta-agent rewrites the actor prompt across episodes, and the outer loop evolves the adaptation policy to maximize Weighted Area Under the Learning Curve. On Jericho ID with GPT-5 as meta-agent, average W-AUC increases from 0.18 for Naive to 0.41 for Meta-TTL (Lou et al., 1 Apr 2026).

These results underline an important expansion of TTL: the adaptation target may be not only model parameters, but also curricula, pseudo-label generation rules, or adaptation policies over repeated episodes.

6. Weight-free textual adaptation, applications, and limitations

Not all TTL methods update weights. TSAN performs test-time preference optimization entirely in natural language. Given a prompt $z_i=[x_i;y_i]$8, it generates multiple candidates $z_i=[x_i;y_i]$9, forms textual keys and values from top-$z=[x;0]$0 candidates, uses an LLM-based attention module to produce a textual attention report, aggregates candidate strengths into a new response, critiques that response in a “textual gradient space,” and iterates. The method is explicitly parameter-free at test time. With three test-time iterations on a base SFT model, the paper reports improvements such as AlpacaEval 2 LC from 3.01 to 18.57 and MATH-500 from 22.0 to 28.2 for the unaligned base model (Mo et al., 10 Nov 2025).

SkillTTA likewise adapts a fixed solver through text rather than weights. It retrieves a small set of relevant training trajectories, synthesizes them into a temporary task-specific textual skill $z=[x;0]$1, and injects that skill into the solver prompt. On SpreadsheetBench, task-specific skills improve Pass@1 from 0.397 to 0.505 relative to static trajectory-to-skill synthesis using GPT-5.5; on BigCodeBench, Pass@1 improves from 0.517 to 0.651 (Wang et al., 16 May 2026).

TTL also appears outside pure language tasks. In CLIP-based OOD detection, "TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-LLMs" updates learnable OOD prompts from pseudo-labeled test samples, uses OOD Knowledge Purification to suppress ID-boundary noise, and stores high-quality textual features in an OOD Textual Knowledge Bank. On ImageNet-1k, the method reports average FPR95 = 12.46 and AUROC = 97.29, compared with AdaNeg at FPR95 = 19.22 and AUROC = 96.17 (Ye et al., 17 Apr 2026).

Several recurrent limitations also emerge. TLM still requires backward passes and threshold tuning, and explicitly identifies forward-pass-only adaptation as future work (Hu et al., 27 May 2025). TTT-NN depends on a sufficiently large and relevant index, with a combined text-plus-vector footprint of about 2.1 TB and added inference-time retrieval and backpropagation cost (Hardt et al., 2023). Passage-specific TTL for reading comprehension raises inference time to about 5 minutes per context for best BERT-Large TTLO and about 1.6 minutes for DistilBERT TTLO (Banerjee et al., 2021). The strongest theoretical guarantees are narrow: linear transformers with one gradient step, or nonlinear single-index models under Gaussian covariates and polynomial links (Gozeten et al., 14 Mar 2025, Kuwataka et al., 30 Sep 2025).

A common misconception is that TTL is synonymous with prompting or with standard in-context learning. The literature draws a sharper boundary. In standard ICL, the model conditions on demonstrations while weights are fixed; in test-time training for ICL, the model updates weights using the in-context demonstrations before final prediction (Gozeten et al., 14 Mar 2025). Another misconception is that stronger external supervision is always preferable: Meta-TTRL for unified multimodal models argues instead for “metacognitive synergy,” where monitoring signals aligned with the model’s own optimization regime outperform stronger but mismatched evaluators (Tan et al., 16 Mar 2026).

Across these works, TTL appears less as a single method than as a design principle: inference-time text, context, trajectories, or interaction histories are treated as a source of adaptation rather than a passive input. The main open technical fault lines concern when parameter updates are necessary, how to control noise from self-generated supervision, how to preserve prior knowledge while specializing locally, and how far current theoretical analyses extend beyond the highly structured regimes in which they are presently proved.