TTT-Discover: Test-Time Training for Discovery

• The paper introduces a novel method that adapts LLMs at test time using an entropic utility objective to discover record-breaking solutions.
• It employs reinforcement learning with adaptive β scaling and a PUCT-inspired strategy to focus exploration on high-reward actions.
• TTT-Discover is applied in fields such as mathematics, GPU optimization, algorithm design, and biology, yielding significant performance gains.

Test-Time Training to Discover (TTT-Discover) is a paradigm for leveraging pretrained models—especially LLMs—to achieve state-of-the-art solutions on individual, hard scientific problems by continuing to train the model at inference time, with the explicit aim of discovering record-breaking solutions rather than merely robust average-case performance. TTT-Discover uniquely combines reinforcement learning, adaptive entropic objectives, and optimistic search algorithms, focusing the model’s capacity on a single targeted discovery task using feedback derived from continuous reward functions. The methodology has been demonstrated to advance the state-of-the-art in diverse domains including mathematics, GPU kernel optimization, algorithm design, and biological data analysis, and is supported by contemporary advances in test-time adaptation, continual learning, and scalable online optimization (Yuksekgonul et al., 22 Jan 2026).

1. Conceptual Foundations and Motivation

Classic test-time adaptation methods, such as those used in robust classification, treat the test environment as a stream of data for model adaptation, typically seeking to reduce average error or improve coverage over possible tasks (Su et al., 2022, Hübotter et al., 29 Sep 2025). In contrast, TTT-Discover is motivated by the demand in scientific discovery and engineering for single-instance breakthroughs: the goal is to find one solution that sets a new record or meets an unprecedented threshold, not exhaustively improve generalization.

Predecessors such as AlphaEvolve treat the LLM as a frozen sampler, performing black-box search via repeated sampling and heuristic replay but leaving the model itself unchanged. TTT-Discover departs fundamentally by casting the single test problem as a unique reinforcement learning (RL) environment and continually adapting the LLM policy on-the-fly, so all gradients and learning signal are problem- and instance-specific. This approach exploits the fact that in hard, out-of-distribution (OOD) problems, feedback from attempted solutions provides the most relevant supervision for further exploration (Yuksekgonul et al., 22 Jan 2026).

2. Formal Objective and Algorithmic Structure

Let $\pi_\theta(a|d, s)$ be the parameterized policy of the LLM, where $d$ is the fixed problem description and $s$ is the current state or candidate (e.g., a partial solution, code fragment, or hypothesis). Each model rollout (action $a$) yields a new state $s'$ and a real-valued, verifiable reward $R(s')$.

Rather than maximizing the expected reward (as in classical policy gradient), TTT-Discover employs an entropic utility:

$$J_\beta(\theta; s) = \log \mathbb{E}_{a\sim\pi_\theta(\cdot|s)}\left[ \exp(\beta R(s,a)) \right]$$

This objective interpolates between expected reward ($\beta\to 0$) and pure best-of-N search ($\beta\to\infty$), ensuring the policy increasingly concentrates probability on actions yielding maximal observed rewards. The gradient is:

$$\nabla_\theta J_\beta(\theta;s) = \mathbb{E}_{a\sim\pi_\theta} \left[ w_\beta(a;s)\, \nabla_\theta \log \pi_\theta(a|s) \right],$$

where $w_\beta(a;s)$ is a softmax-weighted relative reward. To ensure stable adaptation and controlled exploitation/exploration, $\beta$ is adapted online per state by enforcing a KL-divergence budget (i.e., $KL[q_\beta(\cdot|s)\|\pi_\theta(\cdot|s)] = \gamma$), typically by a batchwise line search.

For search efficiency, a PUCT-inspired state selection strategy is used. At each step, an archive $\mathcal{H}_i$ tracks candidate states and their rewards; expansion prioritizes states with high max-reward optimism, high payoff to exploration, and strong prior ranking. Sampling and expansion are thus guided by:

$$\text{score}(s) = Q(s) + c \cdot \text{scale} \cdot P(s) \sqrt{(1+T)/(1+n(s))}$$

with $Q(s)$ the best reward from $s$, $n(s)$ subtree count, $T$ total expansions, and $P(s)$ a rank-based heuristic.

3. Practical Implementation: RL, Optimization, and Infrastructure

TTT-Discover is implemented atop an open 120B-parameter LLM (OpenAI gpt-oss-120b) augmented by LoRA rank-32 adapters for efficient, low-cost parameter tuning. Reinforcement learning proceeds online, iterating over batches of rollouts wherein all candidates share a fixed policy, and updates are performed using Adam (learning rate $4\cdot10^{-5}$, standard momentum). KL divergence penalties regularize movement from the original policy, and per-step temperature scaling (adaptive $\beta$) ensures focus on high-reward solutions while retaining sufficient diversity for discovery.

Each search-train cycle generates 512 rollouts per step, subdivided into groups for context reuse, repeated over 50 training steps—yielding on the order of 25,600 samples per problem. Computational cost is modest, typically between \$300–\$600 per run using the Tinker API, leveraging a 32,768-token context window. All code and discovered solutions are made available for reproducibility and independent verification (Yuksekgonul et al., 22 Jan 2026).

4. Domains and Continuous-Reward Applications

TTT-Discover has been applied successfully to domains characterized by continuous, verifiable reward feedback:

• Mathematics: For problems such as Erdős’ Minimum Overlap and specific autocorrelation inequalities, solutions are numerical constructions verified via Python-generated code and objective calculations on the property of interest (e.g., $1/(\max\,\text{overlap})$).
• GPU Kernel Engineering: For the GPUMode (TriMul and MLA) competitions, candidates are Triton/PyTorch kernels, with rewards measured as $1/\text{runtime}$ on held-out shapes using H100, A100, B200, and MI300X GPUs.
• Algorithm Design: In AtCoder heuristic contest settings, candidate solutions are C++ code, evaluated by official challenge scoring harnesses.
• Biology: For single-cell data denoising, the candidate is a Python analysis pipeline, scored via normalized MSE or combined MSE/Poisson metrics under molecular cross-validation (Yuksekgonul et al., 22 Jan 2026).

Continuous rewards (rather than purely discrete thresholds) enable stable guidance of the entropic policy gradient and permit PUCT-driven search to reliably escalate solution quality.

5. Empirical Achievements and Ablation Findings

TTT-Discover consistently advances state-of-the-art solution quality in its target domains:

• Mathematics: Improved Erdős’ minimum overlap from 0.380924 (AlphaEvolve) to 0.380876 and tightened autocorrelation inequalities, constructing larger and asymmetric solutions beyond prior bests.
• GPU Kernels: Delivered kernels $2\times$ faster than prior art (e.g., 1161μs vs. human 1371μs on H100; 2198μs vs. 4531μs on A100), with strong generalization to unseen hardware.
• Algorithm Design: Surpassed both seeded and from-scratch human winners in AtCoder contests, notably reaching 1st place in ahc039 and exceeding best human results in ahc058.
• Biological Data Analysis: Improved normalized MSE/Poisson scores from 0.64 (MAGIC baseline) to 0.71–0.73 on PBMC/Tabula benchmarks (Yuksekgonul et al., 22 Jan 2026).

Ablation studies demonstrate that the entropic objective, adaptive $\beta$, and PUCT-style state reuse are indispensable: substituting with standard best-of-N search or fixed-temperature exploration incurs substantial performance drops and slows late-stage discovery.

6. Theoretical Perspective and Links to Specialization

TTT-Discover’s effectiveness is grounded in the principle of “specialization after generalization” (Hübotter et al., 29 Sep 2025). Under the linear representation hypothesis (LRH), pretrained foundation models represent a wealth of latent features (“concepts”) in a compressed feature space. Global (frozen) inference averages over these, leading to interference when model capacity $d_2$ is small relative to the concept space $d_1$. By retraining (or adapting a local head) at test time on task-specific signals, TTT-Discover specializes the model capacity to the $s$ relevant concepts for the given problem, achieving a lower test error than possible via global training alone. Empirical analysis with sparse autoencoders and neighborhood-based adaptation strongly supports this theory, with the main benefits realized in the underparameterized regime ($d_2 \ll d_1$) (Hübotter et al., 29 Sep 2025).

7. Broader Context and Extensions

TTT-Discover generalizes the test-time adaptation paradigm beyond classification to complex RL and program synthesis settings, and situates itself among a spectrum of methods such as sequential test-time anchored clustering (TTAC) (Su et al., 2022) and test-time attention-based optimization in vision models (e.g., ViT$^3$) (Han et al., 1 Dec 2025). Anchored clustering demonstrates that adaptation by discovering and aligning latent clusters can yield significant robustness to domain shift, and that pseudo-label filtering, online moment update, and buffer-limited optimization are effective strategies for streaming adaptation—insights compatible with further TTT-Discover advances.

There is ongoing interest in integrating mutual information or contrastive clustering, meta-learned expert heads, and online RL under continual data streams. Core open questions include the optimal allocation of inference budget, managing diminishing returns in the overparameterized regime, and extending to non-stationary and real-time scientific environments.

---

Key References:

• "Learning to Discover at Test Time" (Yuksekgonul et al., 22 Jan 2026)
• "Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering" (Su et al., 2022)
• "Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models" (Hübotter et al., 29 Sep 2025)
• "ViT$^3$: Unlocking Test-Time Training in Vision" (Han et al., 1 Dec 2025)