LiteReason: Efficient Latent Reasoning for LLMs

• LiteReason is a lightweight latent reasoning approach for LLMs that interleaves discrete token generation with continuous latent embeddings to efficiently handle long narrative contexts.
• It employs a dual-mode system with reinforcement learning to optimize the switch between explicit token sampling and latent embedding generation, significantly reducing computational costs.
• Empirical evaluations show that LiteReason nearly matches full reinforcement learning performance in narrative tasks while reducing token usage by up to 92%.

LiteReason is a lightweight latent reasoning approach for LLMs, specifically designed to minimize computational cost while maintaining high reasoning performance on complex tasks involving long narrative contexts. The method introduces an architecture that seamlessly interleaves discrete chain-of-thought token generation with short bursts of continuous latent embeddings, optimizing the overall process by incorporating reinforcement learning (RL) to determine when to switch modes. LiteReason addresses the challenge of expensive inference and RL fine-tuning for narrative tasks where traditional chain-of-thought traces can extend to thousands of tokens, incurring substantial cost in terms of both computation and resources (Gurung et al., 1 Dec 2025).

1. Background and Motivation

Traditional LLMs rely on explicit chain-of-thought (CoT) reasoning, emitting lengthy intermediate traces before arriving at a final answer. While effective in domains such as mathematics and logic, this approach becomes prohibitively expensive on tasks such as plot hole detection in ∼1k-token stories or chapter planning in ∼6k-token books. The high computational and energy costs arise because each additional token in a reasoning trace requires a full forward (and, in RL, backward) pass through a large Transformer.

Latent reasoning methods address this inefficiency by substituting contiguous spans of discrete tokens with continuous “latent tokens”—embeddings that can represent several routine or obvious reasoning steps compactly. Only a subset of the overall reasoning process is represented explicitly; other segments are encoded in these denser latent representations. For narrative generation and comprehension, where human-like implicit reasoning dominates, such a hybrid architecture is especially compelling. LiteReason extends these insights with a system that enables RL-optimized switching between discrete and latent modes, maintaining most of the performance gains of conventional RL fine-tuning while dramatically reducing the number of generated tokens (Gurung et al., 1 Dec 2025).

2. System Architecture

LiteReason involves two core modules: (1) the base LLM (frozen during certain stages), and (2) a lightweight “Reasoning Projector” that outputs continuous latent tokens on demand. The interplay between discrete and latent generation modes is orchestrated through a special tagging system and a custom policy.

2.1 Discrete and Latent Reasoning Flow

1. Discrete mode: Standard token sampling proceeds via the LLM head.
2. Latent mode activation: Upon emission of a special tag, e.g., <implicit_thought>#</implicit_thought>, where # specifies the number of latent steps, the system switches to latent mode for the prescribed number of steps.
3. Latent mode: The last hidden state is passed through the Reasoning Projector, producing a latent embedding $e_t \in \mathbb{R}^d$ (matching token embedding dimension), which is fed into the LLM as input, bypassing discrete sampling.
4. Resumption: After # latent steps, the system switches back to discrete token generation.
5. Repeat until output is complete.

2.2 Reasoning Projector Module

The Reasoning Projector is a compact two-layer MLP:

$$e_t = W_2 \, \mathrm{ReLU}(W_1 h_t + b_1) + b_2;\qquad e_t \in \mathbb{R}^d$$

where $h_t$ is the LLM’s hidden state at a switch point. Only the MLP parameters are subject to supervised tuning; the LLM remains frozen during Projector SFT.

2.3 Control and Interleaving

Switching between latent and discrete modes is governed entirely by the LLM’s emission of special tags. Discrete mode uses ordinary vocabulary tokens; latent mode uses projector outputs for the required number of steps.

3. RL-Based Reasoning Policy and Training Objectives

The system’s policy $\pi_\theta$—the LLM’s token-generation distribution—jointly determines:

• What content to generate (tokens or latent spans)
• When to invoke the Reasoning Projector via <implicit_thought>#</implicit_thought> tags

3.1 Supervised Fine-Tuning (SFT) of the Projector

• Reasoning traces are collected using rejection sampling, accepting only those with non-negative reward.
• In each trace, a proportion $s_r$ of sentences is randomly masked, replaced with latent tags, where the latent span length $#$ is set proportional to sentence length (token replacement ratio $t_r$).
• Training is by cross-entropy on the unmasked part, with only Projector parameters updated.

This process encourages the latent embeddings to bridge masked content, learning to compress and represent implicit reasoning steps.

3.2 RL Fine-Tuning for the Policy

• Objective: maximize expected reward $$J(\theta) = \mathbb{E}_{\tau \sim \pi_\theta} [R(\tau)]$$
• Policy gradients are computed via

$$\nabla_\theta J(\theta) = \mathbb{E}_{\tau \sim \pi_\theta} [R(\tau) \nabla_\theta \log \pi_\theta (\tau)]$$

• Group Relative Policy Optimization (GRPO) is used: for each prompt, $G$ rollouts are sampled and their rewards normalized into advantages before updating $$e_t = W_2 \, \mathrm{ReLU}(W_1 h_t + b_1) + b_2;\qquad e_t \in \mathbb{R}^d$$0.
• Only discrete token actions (including the special tags) receive gradients; the Projector is fixed during RL.

3.3 Combined Objective

The loss is given by

$$e_t = W_2 \, \mathrm{ReLU}(W_1 h_t + b_1) + b_2;\qquad e_t \in \mathbb{R}^d$$1

where $$e_t = W_2 \, \mathrm{ReLU}(W_1 h_t + b_1) + b_2;\qquad e_t \in \mathbb{R}^d$$2 is the set of discrete-sampling actions in trajectory $$e_t = W_2 \, \mathrm{ReLU}(W_1 h_t + b_1) + b_2;\qquad e_t \in \mathbb{R}^d$$3. An optional SFT distillation term maintains Projector alignment.

4. Empirical Evaluation

Evaluation focuses on two narrative reasoning tasks:

• Plot-Hole Detection (Flawed Fictions): binary Yes/No reward.
• Next Chapter Prediction (NCP): improvement in true chapter likelihood (“Contrastive Improvement”).

4.1 Token Compression and Performance

Method	Flawed Fictions Accuracy	Flawed Fictions Avg Tokens	NCP Contrastive Improvement	NCP Avg Tokens
Base LLM	57.3%	400	0.067	831.9
Non-latent RL	88.7%	114.5	0.666	721.3
LiteReason	87.4%	34.0	0.478	193.1

LiteReason achieves comparable accuracy to non-latent RL methods while reducing token count by 77–92%.

4.2 RL Sample Efficiency

Experiment	RL Discrete Tokens	RL + LiteReason Tokens	Reduction
Flawed Fictions	61.4 million	29.0 million	–52.8%
Next Chapter Prediction	108.8 million	54.9 million	–49.5%

LiteReason halves the required number of discrete-token actions during RL.

4.3 Comparison with Prior Latent Reasoning Methods

Method	Accuracy / Contr. Imp.	Avg. Tokens (FF / NCP)
Base LLM	57.3% / 0.067	400 / 832
MoI	58.1% / 0.045	402 / 829
Soft Thinking	57.4% / 0.013	361 / 967
COCONUT	50.7% / 0.083	268 / 362
CoLaR	53.7% / 0.118	226 / 218
LiteReason	87.4% / 0.478	34 / 193
RL upper bound	88.7% / 0.666	115 / 721

LiteReason visibly surpasses previous latent reasoning strategies in both efficiency and performance, recovering approximately 96% of the RL-trained gain in plot-hole detection and 69% in chapter prediction, while emitting far fewer tokens.

5. Performance-Compute Tradeoff and Limitations

5.1 Pareto Frontier Shift

LiteReason delivers near-RL trained performance at a fraction (∼25–30%) of the token computational cost. Plotting accuracy (or Contrastive Improvement) against average token budget reveals a Pareto frontier where LiteReason closely tracks the performance of full RL tuning with a sharply reduced compute footprint.

5.2 Constraints and Research Directions

Several technical caveats and open questions remain:

• Projector staleness: The Reasoning Projector is frozen during RL and only sporadically updated with SFT. More frequent or joint training could further align latent reasoning with evolving LLM behaviors.
• Hyperparameter sensitivity: Performance depends on tuning for token- and sentence-replacement ratios as well as the number of latent steps per task.
• Architectural exploration: Alternative Projector designs (e.g., attention mechanisms, recurrence, persistent latent state) could support richer compressed reasoning.
• Broader applicability: LiteReason has only been validated on narrative domains; extending to modalities such as code or multimodal tasks is an open avenue.

6. Summary and Outlook

LiteReason is a latent reasoning framework for LLMs that interleaves bursts of continuous latent embeddings—output from a lightweight Reasoning Projector—with conventional token generation, using RL to optimize both content and reasoning mode switching. The approach nearly matches the reasoning accuracy of fully discrete RL-trained models on long-form narrative tasks while reducing token usage by 77–92%. This constitutes a shift in the performance–computation tradeoff curve, offering efficient and scalable narrative reasoning for large models (Gurung et al., 1 Dec 2025).