Latent Reasoning: Advances in SFT

• Latent reasoning SFT efforts are a growing research area focused on fine-tuning LLMs and MLLMs to unveil and enhance implicit reasoning capabilities with minimal human-annotated data.
• They incorporate diverse approaches—alternating SFT/RL, unified objectives, and latent variable optimization—to robustly elicit, regularize, and compress reasoning paths.
• Empirical results demonstrate improved accuracy, generalization, and efficiency, achieving state-of-the-art performance with significantly reduced sample and rollout requirements.

Latent reasoning SFT (Supervised Fine-Tuning) efforts encompass a rapidly expanding research area that seeks to efficiently and robustly elicit, regularize, and enhance the reasoning capabilities latent in LLMs and multimodal LLMs (MLLMs). These efforts address both token-space and continuous (latent-space) chains of thought, with a focus on mitigating catastrophic forgetting, improving sample and compute efficiency, and achieving strong generalization with limited human-annotated data. The following sections survey key methodologies, theory, architectures, and empirical findings at the intersection of SFT and latent reasoning.

1. Methodological Paradigms for Latent Reasoning SFT

Latent reasoning SFT methods fall into several classes shaping how explicit or implicit, alternating or unified, the integration of SFT and RL is:

• Alternating/Interleaved SFT and RL: In frameworks such as MIFO (Yuan et al., 6 Oct 2025), SFT and RL phases are interleaved, with SFT applied only to buffered, hard examples from RL, using entropy-based filtering of update targets and parameter freezing to prevent overwriting RL-acquired skills.
• RL-then-SFT: Metis-RISE (Qiu et al., 16 Jun 2025) omits the standard cold-start SFT, using RL to first activate latent reasoning (with group-relative advantage normalization and asymmetrically clipped objectives), followed by targeted SFT to both self-distill successful explorations and augment with expert traces where reasoning is absent.
• Single-Stage Unified SFT-RL: Methods such as SRFT (Fu et al., 24 Jun 2025) forgo strict separation, instead optimizing both MLE (supervised) and RL-inspired objectives simultaneously, dynamically weighting each by entropy, thus maintaining plasticity during training and resisting catastrophic forgetting.
• Latent Variable and Self-Rewarding SFT: LaTRO (Chen et al., 6 Nov 2024) casts reasoning as latent variable optimization via an amortized ELBO, self-rewarding the model on its own log-likelihood, and unifying the MLE and policy-gradient terms into a single variational fine-tuning pipeline without the need for external rewards or rationale annotations.
• Latent Token Superposition and Compression: Latent reasoning methods such as Latent-SFT (Deng et al., 17 Oct 2025) employ specialized attention masks and supervision strategies that force the model’s latent reasoning trajectory to lie in a well-structured, vocabulary-aligned subspace, achieving high compression rates and emergent multipath reasoning parallelism.
• Automatic Rationale Discovery via Pattern-Aware SFT: For pattern-centric reasoning problems, PARO (Pang et al., 14 Oct 2025) demonstrates that LLMs can be prompted to generate self-consistent, pattern-aligned rationales requiring as little as two human exemplars, with subsequent SFT matching or exceeding human-annotated SFT in efficacy under RL-based reward fine-tuning.
• Pure Online/Autoregressive SFT: OSFT (Li et al., 21 Oct 2025) performs reward-free, self-generative SFT in which the model generates outputs for real prompts and is immediately fine-tuned on its own responses, reinforcing existing latent knowledge and achieving RL-matched performance on math reasoning with one rollout per prompt.

This diversity of approaches reflects both the variety of application domains (language, vision, multi-modal) and the spectrum of available supervision (from full gold chains to reward-only, to no external signal at all).

2. Strategies for Catastrophic Forgetting and Efficiency

Latent reasoning SFT research devotes substantial attention to preventing catastrophic forgetting, amplifying data efficiency, and tailoring updates:

• Selective SFT via High-Entropy Token Selection: In MIFO, SFT gradients are concentrated on token positions where the model is most uncertain (as measured by output entropy relative to an example-specific quantile), halving update magnitudes and reducing overfitting.
• RL-Critical Parameter Freezing: By tracking parameter update norms across RL phases and constructing a decayed importance map, MIFO freezes the top fraction of RL-critical weights during SFT to preserve the RL-acquired policy, resuming full parameter updates only after SFT completes.
• Accuracy-Driven Buffering: SFT is only applied to examples with rollout accuracy below a small threshold, ensuring that the improvement effort is targeted where RL alone is ineffective or sparse in signal (Yuan et al., 6 Oct 2025, Zhang et al., 20 Jun 2025).
• Curriculum via Branched Rollouts: BREAD (Zhang et al., 20 Jun 2025) employs partial expert prefixes (branched rollouts) to scaffold intermediate successes, densifying the RL reward signal and forming an adaptive curriculum that outperforms both SFT+RL and pure RL on small model regimes with dramatically lower expert trace usage.
• Hybrid Variational and Contrastive Objectives: LTA-Thinker (Wang et al., 16 Sep 2025) leverages a learnable transformer prior that boosts variance in latent thought generation, jointly optimized with semantic alignment (KL-divergence anchoring to question semantics) and reasoning focus (contrastive) losses, increasing information efficiency and the attainable scaling ceiling.
• Self-Rewarding and Self-Distillation: OSFT and LaTRO show that models can reinforce their existing latent chains either by online SFT on self-sampled traces or by using their own prediction likelihoods as the self-reward, obviating the need for extrinsic rewards or curated rationales (Li et al., 21 Oct 2025, Chen et al., 6 Nov 2024).

Efficiency gains are substantial: MIFO achieves state-of-the-art reasoning using only 1.5% of the SFT examples and 20.4% of RL rollouts needed by prior leading methods (Yuan et al., 6 Oct 2025), and BREAD solves previously unsolvable regimes using under 40% of the expert traces (Zhang et al., 20 Jun 2025).

3. Theoretical Insights: Overfitting, Diversity, and Exploration

Careful analysis reveals several key theoretical and empirical results shaping best practices:

• SFT Score Is Not a Reliable Predictor of RL Gains: High SFT pass@1 is often uncorrelated or even negatively correlated with post-RL performance, due to overfitting on easy or homogeneous data (Kang et al., 2 Oct 2025). Instead, generalization loss (held-out cross-entropy) and pass@k metrics at large k more accurately predict RL outcome.
• Data Diversity and Over-Training: Increasing SFT epochs or focusing on short/repeated examples inflates SFT pass@1 but curtails RL gains and generalization. Optimal pipelines favor diverse, unique, and varied-length samples, stopping SFT early as measured by rising generalization loss (Kang et al., 2 Oct 2025).
• Single-Stage vs Interleaved Optimization: Unified SRFT (Fu et al., 24 Jun 2025) demonstrates that a single-stage, entropy-weighted mixture of SFT and RL objectives navigates probability space more directly to high-performing regimes, avoiding the overshooting and subsequent reversal (catastrophic forgetting) seen in multi-stage methods.
• Entropy as an Intrinsic Signal: SRFT uses policy entropy to balance when to trust demonstrations (high-entropy policy, SFT weight up) and when to exploit rewards (low entropy, RL weight up), curbing premature entropy collapse (which manifests as mode-seeking and poor generalization).
• Latent Space Regularization: SFT alone can induce large representation (PCA) and output distribution (KL-divergence, rank shifts) drifts, leading to catastrophic forgetting of domain-general capabilities. RL and hybrid methods retain much tighter alignment with base geometry, preserving non-math task competence while raising math task accuracy (Huan et al., 1 Jul 2025).

4. Architectures and Supervision Mechanisms for Latent Reasoning

Architectural approaches to latent reasoning SFT span discrete and continuous representations, as well as curriculum and structural guides:

• Latent Token Superposition: Latent-SFT (Deng et al., 17 Oct 2025) enforces that latent tokens live in the low-rank column space of the vocabulary embedding, and uses KL+CE objectives with tailored attention masks to compactly summarize explicit reasoning paths. This supports compression rates up to 4x, maintaining or exceeding explicit CoT accuracy.
• Learnable Transformer Prior: LTA-Thinker (Wang et al., 16 Sep 2025) employs a randomly initialized, learnable transformer to generate high-variance latent thought vectors, regularized via KL alignment to question semantics and contrastive learning focused on reasoning steps most influential for answer prediction.
• Vision-Latent CoT: Monet (Wang et al., 26 Nov 2025) distills intermediate visual evidence into continuous latent embeddings via a three-stage SFT pipeline (warm-up on image-text interleaved CoT, supervised image-to-latent alignment, and latent-only generation). Specialized attention masks and latent-only gradient flow ensure effective supervision.
• Adaptive Latent Step Sizing: Recent adaptive SFT pipelines (Ning et al., 26 Nov 2025) introduce architectures allowing models to learn when to stop iterating latent steps based on a learned “stop” head, with RL post-SFT optimizing for minimum reasoning length under accuracy constraints.
• Automatic Rationale Discovery: Pattern-aware pipelines such as PARO (Pang et al., 14 Oct 2025) show that LLMs require only a minimal set of pattern exemplars to synthesize rationales that can replace costly human annotations, provided the underlying reasoning structure is stable across tasks.

5. Empirical Performance and Benchmarking

Empirical investigations across benchmarks such as GSM8K, ARC-Challenge, MATH500, and OpenCompass demonstrate consistent gains from latent reasoning SFT innovations:

Method/Framework	SFT Data Used	RL Data Used	Pass@1 / Benchmark Gains	Key Result
MIFO (Yuan et al., 6 Oct 2025)	1.5%	20.4%	State-of-the-art	Shortest traces, no forgetting
Metis-RISE (Qiu et al., 16 Jun 2025)	—	~40K samples	+4–5 pts (RL), +1–2 pts (SFT)	SoTA on OpenCompass-MMR
LaTRO (Chen et al., 6 Nov 2024)	Full SFT	—	+12.5% (GSM8K)	Requires no annot. rewards
BREAD (Zhang et al., 20 Jun 2025)	<40%	—	+11% on NMC	Curriculum, 3x speedup
SRFT (Fu et al., 24 Jun 2025)	Full	Full	59.1% (avg), +4.8% over SFT	SoTA, OOD generalization
LTA-Thinker (Wang et al., 16 Sep 2025)	Full	—	+2.05–+0.59% over SoftCoT++	SOTA at low sample count
Latent-SFT (Deng et al., 17 Oct 2025)	Full	—	Matches explicit SFT	4x chain compression
Monet (Wang et al., 26 Nov 2025)	Full (MLLM)	Full (w/ VLPO)	+4–9 pt real, +2–3 pt OOD	First explicit visual latent RL
OSFT (Li et al., 21 Oct 2025)	None (self)	None	Comparable to RLVR	Reward-free, 1 rollout

These methods report not only higher absolute accuracy but also improved sample efficiency, reduced reasoning trace length, and enhanced out-of-distribution (OOD) generalization.

6. Limitations, Controversies, and Prospective Directions

Despite these advances, several limitations and open challenges persist:

• Catastrophic Domain Forgetting: SFT alone on narrow domains (e.g., math) can catastrophically erase general-domain skills, as shown by large PCA and KL shifts; RL-based or entropy-aware hybrid approaches mitigate but do not entirely eliminate the challenge (Huan et al., 1 Jul 2025, Kang et al., 2 Oct 2025).
• Data and Curriculum Design: The efficacy of these methods heavily depends on the diversity and difficulty calibration of training data. Automated curriculum generation (e.g., Episode Anchor Search in BREAD) and pattern-induced supervision (as in PARO) are promising, but require task-specific engineering and domain insight (Zhang et al., 20 Jun 2025, Pang et al., 14 Oct 2025).
• Scaling and Generalization: Although LTA-Thinker and Latent-SFT demonstrate strong scaling effects and sample efficiency, more research is needed on extending these frameworks to adaptive reasoning lengths, variable latent-space dimensionality, and hyperparameter robustness (Ning et al., 26 Nov 2025, Wang et al., 16 Sep 2025).
• Architectural Innovations: Further work is necessary to refine latent-state recurrent architectures, latent head classifiers, and continuous-action RL objectives (e.g., VLPO), especially for high-dimensional or truly multi-modal latent reasoning (Wang et al., 26 Nov 2025, Ning et al., 26 Nov 2025).
• Evaluation Metrics: Reliance on pass@1 or accuracy measures can obscure real-world utility. Generalization loss and pass@large k provide better proxies for downstream generalization, and representations shifts in both latent and token spaces offer important diagnostics (Kang et al., 2 Oct 2025).
• Human-in-the-Loop and Pattern Discovery: Minimal rationales (PARO) suffice only when the reasoning structure is fixed; generalizing to compositional or adaptive reasoning tasks remains an open frontier (Pang et al., 14 Oct 2025).

A plausible implication is that future latent reasoning SFT pipelines will combine dynamic, entropy-aware, and variationally regularized objectives with adaptive curriculum strategies, while employing architectural components explicitly tailored to both discrete and continuous latent representations and enabling fully autonomous discovery and compression of reasoning paths.

---

References (selected): (Yuan et al., 6 Oct 2025, Qiu et al., 16 Jun 2025, Chen et al., 6 Nov 2024, Ou, 3 Sep 2025, Deng et al., 17 Oct 2025, Zhang et al., 20 Jun 2025, Li et al., 21 Oct 2025, Pang et al., 14 Oct 2025, Bertolazzi et al., 17 Jun 2024, Kang et al., 2 Oct 2025, Wang et al., 26 Nov 2025, Fu et al., 24 Jun 2025, Ning et al., 26 Nov 2025, Wang et al., 16 Sep 2025, Huan et al., 1 Jul 2025)