Reference-state matching in partially observable or irreversible environments

Extend snapshot-based reference-state matching for Explorer-Zero’s environment-interaction verification to partially observable or irreversible environments, where the environment cannot necessarily be reset or its state fully recorded and compared.

Background

Explorer-Zero verifies proposer-generated tasks and intermediate subtasks by comparing solver trajectories with reference states recorded during proposer interaction. This mechanism assumes that the environment can be reset and that relevant state information can be recorded and compared reliably.

The paper notes that this assumption is satisfied by the simulator-based benchmarks studied, but not by settings such as the live web. Extending the verification mechanism to partially observable environments, where the full state may be inaccessible, or irreversible environments, where reset-based comparison may be impossible, is explicitly identified as unresolved.

References

Extending reference-state matching to partially observable or irreversible environments is an open problem.

— Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards  (2610.19868 - Liu et al., 14 Oct 2026) in Appendix, Section 'Limitations' (Section \ref{app:limitations})