Principled Thoughts for Latent Recursive LLM Systems

Abstract: LLMs can reason in continuous space instead of decoded text, by recurring on their own hidden states or by passing those states between agents, while training supervises only the Cross-Entropy (CE) of the final decoded answer and does not constrain the thought. Theoretical and empirical analyses establish and confirm four failures of CE-only training that lead to a lower probability of the correct answer such as collapsing thoughts across distinct questions and retaining irrelevant information. We introduce REST (REpresentation-Supervised Thoughts), a training objective that turns four properties of a valid thought representation (causality, minimality, separability, and stability) into differentiable losses added to CE. We instantiate it in latent single-agent and multi-agent systems, without architectural changes or added parameters at inference. Across 7 benchmarks spanning mathematics, science, medicine, and code generation, with the same training data, compute, and latent budget, REST increases accuracy over CE-only training across agent settings and model sizes by up to 7.5 percentage points and convergence on a final answer by 30\%. Furthermore, REST thoughts encode more of what is required to achieve the correct answer, and decoding them better recovers the intended output of the agent, which makes latent communication easier to interpret. Project Website: https://fard-lab.github.io/REST

• The paper introduces the REST (REpresentation-Supervised Thoughts) method, which adds differentiable surrogates for causality, minimality, separability, and stability loss to the final-answer CE objective in latent recursive LLM systems.
• The REST method improves accuracy by 6.5 points in single-agent systems and 7.5 points in multi-agent systems, demonstrating better representation supervision compared to CE-only training.
• REST enhances the ability of LLMs to maintain accurate and useful semantic information across recursive layers, resulting in more robust and adaptable problem-solving systems.

Problem formulation and central claim

Latent recursive LLM systems replace explicit textual reasoning with continuous representations that are recursively consumed by the same model or transferred between heterogeneous agents. Their standard training signal is the CE of the final answer. The paper argues that this objective is insufficient because it constrains only the downstream output distribution, not the latent thought representation that mediates multiple recursive hops.

The central claim is that CE-only training admits four structurally defective solutions:

1. Causality failure: the thought does not preserve the producer’s output information when substituted for text.
2. Minimality failure: the thought retains information from the producer’s input that is irrelevant to its output.
3. Separability failure: thoughts for semantically unrelated examples collide in representation space.
4. Stability failure: the thought represents one realized output rather than the producer’s uncertainty over possible outputs.

The proposed method, REST (REpresentation-Supervised Thoughts), adds differentiable surrogates for these four properties to the final-answer CE objective. The method is evaluated in both a single-agent self-recursive system and a planner–refiner–solver multi-agent system. The paper reports improvements of up to 7.5 percentage points in multi-agent accuracy and 6.5 points in single-agent accuracy, with matched training data, compute, and latent budgets (2609.36159).

The paper’s strongest conceptual assertion is therefore not merely that auxiliary supervision improves optimization, but that final-answer accuracy is an incomplete diagnostic for latent systems: a low CE can coexist with collapsed, redundant, or unstable intermediate representations.

Latent recursive architecture

The experimental system freezes the base LLMs and an inner link that maps hidden states back into the model’s own embedding space. A latent thought is produced by applying a trainable outer link to the inner-link output:

$$\mathbf{T} = \mathcal{R}_{\psi}\big(\mathcal{R}_{\mathrm{in}}(H_u)\big).$$

At training time, $H_u$ is obtained from a teacher-forced producer pass over a reference output. At inference time, the producer instead performs a fixed number of latent steps, with each hidden state remapped into the next input embedding. The resulting thought has a fixed latent budget, set to 32 steps at inference.

In the multi-agent configuration, a planner produces a latent plan, a refiner transforms it, and a solver generates the final answer. The solver’s thought may be recursively transferred back to the planner for additional rounds. In the single-agent configuration, the same model acts as both producer and consumer, repeatedly conditioning on its own latent state.

REST changes neither the inference architecture nor the number of inference parameters. Only the outer link $\mathcal{R}_{\psi}$ and the auxiliary pooling probes are trained. This design isolates the contribution of representation supervision, but also limits the scope of the empirical claim: the reported gains demonstrate that the outer communication map is underconstrained by CE, not that REST has been optimized jointly with the base models or the inner latent-transition mechanism.

Figure 1: REST augments the final-answer CE objective with representation-level losses applied at every latent transfer.

Why final-answer CE is insufficient

The paper formalizes the cost of substituting a latent thought for a producer’s text. Let $P(\cdot \mid E(u))$ denote the consumer’s output distribution when it receives the producer’s text and $P(\cdot \mid \mathbf{T})$ the distribution when it receives the latent thought. Their divergence determines the loss in probability assigned to a target after substitution. For a target of length $n$, the probability ratio is exponentially sensitive to the per-token discrepancy:

$$P(v \mid \mathbf{T}) = P(v \mid E(u))\exp\big(-n\Lambda(v)\big).$$

Thus, even a modest mismatch per token can substantially reduce the probability of a correct answer, and the mismatch compounds across recursive transfers. CE on the final answer does not directly penalize this intermediate substitution gap.

The four failures arise because the CE objective is evaluated independently on each example and only through the final answer. If two examples produce nearly identical thoughts, CE can interpret the resulting loss as ordinary example difficulty rather than as a representation collision. If the consumer can solve an example without using the thought, the gradient through the thought can become weak, allowing the transfer map to discard producer information or encode redundant input context. Similarly, a single sampled producer output does not specify the producer’s full output distribution, so CE does not require the latent representation to preserve uncertainty.

Figure 2: The four representation failures induced by answer-only CE and the corresponding REST properties.

The theoretical analysis connects these failures to downstream answer probability under explicit assumptions. In particular, the causality analysis assumes bounded disagreement between textual and latent next-token distributions and faithfulness of the reference target distribution to the consumer’s distribution under the textual transfer. Under these conditions, the causality surrogate tracks the sequence-level divergence up to an error proportional to the reference-distribution mismatch. This is a meaningful result, but its quantitative usefulness depends on assumptions that may be weak in practice for heterogeneous agents and teacher-forced training data.

REST objective

REST defines a total objective of the form

$$\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \beta \frac{1}{|\mathcal{H}|} \sum_{(r,k)\in\mathcal{H}} \mathcal{L}_{\mathrm{prop}},$$

where $\mathcal{H}$ indexes latent transfers across agents and recursion rounds. The property loss may contain one or more of four terms.

Causality

Causality penalizes divergence between the consumer’s token distributions under the producer’s text and under the latent thought:

$$\mathcal{L}_{\mathrm{caus}} = \frac{1}{n} \sum_t D_{\mathrm{KL}}\!\left( p(\cdot\mid v_{<t},E(u)) \;\|\; p(\cdot\mid v_{<t},\mathbf{T}) \right).$$

The loss is evaluated under teacher-forced prefixes. The paper proves that the corresponding sequence-level divergence factorizes into an expectation of per-position divergences, so the only approximation is that prefixes are drawn from the training distribution rather than from the consumer’s own textual generations. This makes causality the most directly grounded of the four surrogates.

Minimality

Minimality is implemented as

$H_u$0

where $H_u$1 is the producer input and $H_u$2 its output. The first term preserves output-relevant information, while the negative second term discourages the thought from retaining information about the input once the output is known. This is an operational approximation to an information-bottleneck objective.

The term is particularly well motivated in the single-agent setting. Because the consumer already receives the original question and instructions, any input information redundantly encoded in the thought consumes latent capacity that could otherwise represent intermediate reasoning. The paper reports that minimality is the strongest single property in the Light single-agent system and improves every reported task there while reducing total generated tokens.

The derivation of minimality is less assumption-free than the causality derivation. It requires nested probe classes and bounds on decoder gaps and realization leakage. Moreover, the negative reconstruction term can encourage the representation to discard information that is useful but not captured by the chosen output variable $H_u$3. Consequently, “minimality” is relative to the producer-output factorization and the consumer probes, rather than an intrinsic property of the latent state.

Separability

Separability pools the latent sequence into a vector and penalizes similarity to the pooled representations of preceding examples:

$H_u$4

The term is a contrastive geometric constraint. The paper derives a lower bound on the distance between normalized pooled thoughts as a function of this loss. The derivation assumes that distinct training examples have disjoint semantic supports. This assumption is strong: distinct questions can share solution methods, conclusions, or semantically equivalent reasoning trajectories. The method therefore risks treating legitimate representational reuse as collision unless the negative-sample construction is carefully controlled.

Stability

Stability trains a probe $H_u$5 to predict the producer’s mean token-level predictive entropy:

$H_u$6

The entropy target is estimated from one teacher-forced producer output. The paper proves that this estimator is unbiased when the output is sampled from the producer distribution, and bounds the discrepancy under a producer-reference-faithfulness assumption. In practice, however, the implementation uses a cheap scalar entropy summary rather than a full representation of the output distribution. This is a principled compromise under compute constraints, but it means that the empirical stability claim concerns uncertainty magnitude, not complete distributional preservation.

Empirical results

The evaluation spans MATH500, AIME2025, AIME2026, GPQA-Diamond, MedQA, and code-generation tasks using MBPP+ or LiveCodeBench-v6. The systems use heterogeneous open-weight models between approximately 1B and 4B parameters. Training uses the same Sequential-Math data, 20,000 optimization steps, batch size 4, and matched latent budgets across REST and CE-only conditions.

The principal results are summarized below.

Setting	CE-only reference	Best REST improvement	Strongest configuration
Single-agent, Light	baseline	+3.0 average points	Minimality
Single-agent, Scaled	baseline	+6.5 average points	Minimality
Multi-agent, Light	baseline	+3.8 average points	Causality
Multi-agent, Scaled	baseline	+7.5 average points	Best pair

The gains are not uniform across properties. In the multi-agent Scaled system, the best-pair configuration reaches 87.0% on MATH500 versus 86.6% for CE-only, 83.3% versus 80.0% on AIME2025, 86.7% versus 60.0% on AIME2026, 60.1% versus 59.1% on GPQA-Diamond, 85.0% versus 79.7% on MedQA, and 41.7% versus 33.5% on code generation. These aggregate improvements are substantial, although the very large AIME2026 difference indicates that benchmark-level variance and recursion-round selection materially influence the reported average.

Figure 3: REST improves accuracy while changing the geometry, output preservation, answer convergence, and superposition of latent thoughts.

The paper reports that every single property improves over CE-only in the Light single-agent setting. Minimality gives a +3.0-point average gain in that setting, while the Scaled single-agent configuration obtains its largest improvement from minimality at +6.5 points. In the multi-agent configuration, causality and minimality are consistently reliable; separability and stability are less robust, particularly for the Scaled system. The full weight sweep shows that causality and minimality improve over CE-only across all evaluated $H_u$7 values, whereas separability and stability can reduce accuracy while substantially reducing token usage.

REST also outperforms adapted CODI and SIM-CoT auxiliary losses. In the Light system, REST achieves a +3.8-point average gain over CE-only, compared with +3.3 for CODI and +3.2 for SIM-CoT. In the Scaled system, REST produces a +3.1-point gain, while CODI is nearly neutral and SIM-CoT decreases accuracy. This comparison supports the paper’s distinction between supervising a representation’s relation to a target text and supervising functional properties of the representation itself. It does not, however, establish that REST is universally superior: the baselines are adapted to the same architecture and evaluated under a limited hyperparameter sweep.

Representation-level analyses

The representation analyses provide the paper’s most important evidence beyond benchmark accuracy. PCA projections show that CE-only thoughts form dense clusters even when the corresponding questions are unrelated. In one inspected cluster, the examples concern trigonometric series, prism refraction, bound states in quantum mechanics, combinatorial game theory, recurrence relations, and real-number constructions. The six examples occupy only 2.8% of the projected range under CE-only, compared with 56.4% under separability supervision. Their mean pairwise TF-IDF cosine similarity is 0.03, close to 0.02 for random groups of the same size, indicating that the geometric clustering is not explained by textual similarity.

Figure 4: PCA reveals dense CE-only thought clusters and broader REST representations across semantically unrelated examples.

REST also changes what is decodable from the thought. Relative to CE-only, REST representations contain more content from the producer’s output and less redundant content from the producer’s input. Replacing oracle text with the learned thought recovers more of the solver’s accuracy under REST, directly supporting the claim that the representation preserves information useful to the consumer.

The relationship between training CE and test accuracy is explicitly contradictory to a common optimization assumption. CE-only can achieve lower training CE than causality-supervised models while producing lower test accuracy. In the Scaled system, CE-only memorizes the training set and still underperforms REST. Early stopping CE-only at the point where REST’s CE plateaus does not recover REST’s accuracy. The implication is specific: lower CE is not a sufficient proxy for the quality of latent communication, even when the final answer is the nominal training target.

Figure 5: Training CE can be lower under CE-only without yielding higher downstream accuracy.

The paper further reports a 95% boxed-answer rate for REST versus 73% for CE-only in the analyzed multi-agent setting. REST uses 15.4% more output tokens on average, but the additional tokens are associated with a higher probability of reaching a final answer rather than with repeated text. In the single-agent analysis, CE-only frequently stalls through repeated spans, especially on MedQA, where repeated text occurs in 51.5% of the relevant CE-only samples versus 8.9% under REST. Thus, REST’s token overhead has two distinct sources: it reduces pathological repetition in some single-agent outputs while inducing more genuine derivation in some multi-agent outputs.

Figure 6: REST changes decoded thought content, oracle-text recovery, final-answer convergence, and effective superposition.

The superposition analysis finds that REST preserves or slightly increases the number of candidate reasoning paths supported by a thought. This result is relevant to stability, but the metric is indirect: it decodes latent vectors through the consumer vocabulary and estimates a posterior over candidate plans. It should therefore be interpreted as evidence of multi-hypothesis support under a particular decoding-based operationalization, not as a direct measurement of the full producer distribution.

Recursion depth, scale, and latent budget

REST’s benefit depends on model scale and recursion depth. Additional rounds improve REST relative to CE-only for the Scaled systems but reduce the gain for the Light systems. The paper interprets this as evidence that larger agents can exploit repeated latent exchange, whereas smaller agents obtain most of their benefit in a single round.

The latent-budget experiment provides a related result. At small budgets, REST and CE-only remain close. As the budget increases, CE-only loses accuracy on GPQA-Diamond and LiveCodeBench, while REST maintains its performance. On MATH500, the two objectives remain close across budgets. This suggests that representation supervision becomes more important when the latent channel has sufficient capacity to support degenerate solutions, although the budget analysis consists of single runs per condition and therefore has higher variance.

Figure 7: REST is more stable than CE-only as the latent budget increases in the Scaled system.

The recursion results also qualify any simple claim that more latent computation is uniformly beneficial. The same objective behaves differently across model scale, recursion depth, and agent composition. In particular, adding recursion can amplify REST’s gains for larger systems while diminishing them for smaller ones.

Limitations and open questions

The evaluation trains only the outer link and leaves the base LLMs and inner links frozen. This isolates the auxiliary losses but does not test whether the proposed properties remain effective when the latent transition mechanism and model representations co-adapt. The paper also does not exhaustively sweep all combinations of properties and hyperparameters because the resulting experiment count is multiplicative.

The stability objective is only a scalar entropy-probe approximation to distributional stability. It does not require the thought to preserve the structure of alternative outputs, merely an estimate of their uncertainty. Separability depends on negative examples being semantically disjoint, and the paper’s assumption can fail for related questions or tasks with shared solution structure. Minimality depends on the selected input and output variables and on nested probe classes; different probes could yield different conclusions about what is “irrelevant.”

The experiments average three seeds and report standard errors of approximately $H_u$8 accuracy points and $H_u$9 in token usage for the average-change metric, but the AIME benchmarks remain sensitive to individual problems. Some comparisons also select the best recursion round separately for each system, with the CE-only baseline evaluated at the corresponding round. This is appropriate for matched-round comparison but makes the headline average less representative of a single fixed deployment configuration.

Specific open questions remain. Does REST continue to improve performance when the inner link and base LLMs are trained jointly? Can stability be upgraded from entropy prediction to direct distributional matching without sacrificing the compute advantage? Can separability be defined relative to semantic equivalence classes rather than example identity? Finally, is the observed scale-dependent interaction with recursion an optimization effect, a property of the latent channel capacity, or a consequence of the frozen-model composition?

Conclusion

“Principled Thoughts for Latent Recursive LLM Systems” argues that latent recursive systems require supervision at the representation level, not only at the final answer. REST operationalizes causality, minimality, separability, and stability as differentiable losses added to CE, without architectural changes or additional inference parameters. Across seven benchmarks and both single- and multi-agent systems, the method improves accuracy by up to 7.5 points, increases recovery of producer information, reduces thought collapse, and raises the rate at which systems converge to final answers.

The empirical evidence supports the paper’s principal conclusion: CE-only optimization can produce latent thoughts that achieve low training loss while remaining poor communication representations. REST provides a concrete framework for diagnosing and correcting this mismatch, although the strength and generality of its theoretical guarantees remain conditional on reference-faithfulness, probe, smoothness, and semantic-disjointness assumptions (2609.36159).