Extend REST training beyond the outer link

Extend REST training to the inner link and the agent itself, beyond the current setting in which only the outer link is trained while the base large language models and inner link remain frozen.

Background

REST (REpresentation-Supervised Thoughts) trains the outer link that transfers latent thoughts between agents or recursion rounds, while the base LLMs and the inner link remain frozen. This design isolates the contribution of the four representation-level property losses—causality, minimality, separability, and stability—but leaves open whether the same supervision can be effectively applied to the inner link or to the language-model agents themselves.

The unresolved extension is important because jointly training additional components could alter how latent thoughts are formed, transferred, and consumed, potentially changing the empirical benefits and theoretical behavior reported for outer-link-only training.

References

Extending REST to the training of the inner link or the agent is left to future work.

— Principled Thoughts for Latent Recursive LLM Systems  (2609.36159 - Seddik et al., 28 Sep 2026) in Section 'Conclusion', subsection 'Limitations'