Determine whether the latest continuous thought suffices for long-horizon reasoning

Determine whether the latest continuous thought in Abstract Token Curriculum can summarize all information needed to proceed on long-horizon complex reasoning tasks such as mathematical proof construction, or whether the model must selectively retrieve earlier continuous thoughts, and characterize how this requirement depends on the number of interacting subproblems.

Background

The paper’s experiments suggest that, for the studied parity, graph-reachability, and arithmetic tasks, the final continuous thought can serve as an effective summary for producing the answer, while earlier thoughts are needed to construct subsequent thoughts. The authors identify a potential limitation when reasoning becomes longer and involves several interacting subproblems or revisions.

The conclusion explicitly identifies long-horizon complex reasoning, including mathematical proof construction, as an unresolved setting. The open issue is whether the most recent latent thought remains sufficient or whether effective reasoning requires selective access to earlier latent states.