Programmatic evaluation for non-additive or revision-based action spaces

Develop deterministic programmatic evaluators that can assess intermediate reasoning states and final outputs in non-additive or revision-based action spaces within the STATe-of-Thoughts framework, where actions do not compose via prefix concatenation and partial trajectories cannot be validated prefix-monotonically.

Background

The paper introduces programmatic evaluators as deterministic scoring functions that verify constraint satisfaction for intermediate and final states, which is feasible when action spaces are additive—i.e., each reasoning step produces text that concatenates into a single artifact, enabling prefix-monotonic validation. In such settings, the same function can serve as both a process and outcome reward, allowing invalid trajectories to be pruned immediately.

However, many practical action spaces (e.g., metacognitive operations like self-reflection or targeted editing) are non-additive and do not produce a single, incrementally valid artifact at each step. The authors note that programmatic evaluation in these contexts is currently not supported and identify extending it to non-additive or revision-based action spaces as an unresolved challenge.