Revisiting Open-Loop Execution in Robotics: Toward Reactive, Higher-Performing Policies

Abstract: Action chunking --- the practice of predicting a sequence of actions and executing a prefix open-loop --- has emerged as a key enabler of recent progress in imitation learning for robotic manipulation. However, executing long open-loop prefixes reduces reactivity, limiting policies' ability to correct for errors. Further, the mechanisms underlying these performance benefits remain poorly understood: prior works cite mitigating compounding errors, absorbing inference latency, or smoothing motions, but provide limited controlled evidence or guidance for preserving reactivity. In this work, we argue that long open-loop execution primarily helps short-context policies imitate "non-Markovian demonstrations". Across four simulation and two real-world tasks, we show that expert non-Markovianity strongly shapes the relationship between task success and open-loop execution horizon. Further, we investigate the impact of compounding errors --- the prevailing explanation for long open-loop execution in prior work --- and find that while they matter, expert non-Markovianity has a much stronger impact in our experimental setting. Finally, we show that when policies are provided with a sufficiently long context, open-loop execution is no longer beneficial and the most reactive, closed-loop policies perform best. While imitation learning has seen great success using long open-loop execution, our findings motivate long-context, reactive policies as a more principled and performant paradigm.

• The paper shows that policies trained on non-Markovian demonstrations develop inverted-U-shaped success-horizon curves, while Markovian demonstrations generally favor one-step reactive execution.
• The paper finds that long action chunks reduce hidden-state aliasing and cycling by limiting replanning, but they trade ambiguity reduction for weaker responses to disturbances and contact changes.
• The paper demonstrates that longer observation context can shift optimal execution toward one or two actions and improve performance, reaching 93.2% versus 90.6% in simulation and up to 90% in real-world trials.

Central thesis and problem formulation

“Revisiting Open-Loop Execution in Robotics: Toward Reactive, Higher-Performing Policies” (2608.15938) examines why imitation-learning policies often achieve higher task success when they execute predicted action sequences open-loop for multiple timesteps rather than replanning after every action. The paper challenges the prevailing interpretation that long execution horizons are primarily required to mitigate compounding errors, smooth action transitions, or absorb inference latency. Its central claim is more specific: long open-loop execution mainly compensates for the inability of short-context policies to imitate non-Markovian expert demonstrations.

The distinction is between the prediction horizon and the execution horizon. A Diffusion Policy predicts an action sequence of fixed length, but executes only its first $T_{\mathrm{exec}}$ actions before acquiring new observations and replanning. The paper holds the prediction horizon fixed while varying $T_{\mathrm{exec}}$, thereby isolating the effect of reactivity. The resulting relationship between success rate and execution horizon is termed the success-horizon curve. The familiar pattern is often inverted-U-shaped: very short execution horizons perform poorly, intermediate horizons perform best, and very long horizons again degrade because the policy reacts too slowly.

The paper argues that this curve is not an intrinsic property of action chunking. Rather, it depends strongly on the temporal structure of the demonstrations and on the policy’s available observation history. A Markovian expert selects actions from the current environment state, whereas a non-Markovian expert’s action distribution depends on information accumulated over the trajectory. Human teleoperators are treated as inherently non-Markovian because their decisions may depend on latent intentions, elapsed time, prior actions, pauses, and unrecorded perceptual judgments. A policy receiving only one or two frames cannot generally reconstruct these variables.

The experimental scope is broad. The authors study four simulated manipulation tasks—FurnitureSimOneLeg, PushT, GearInsertion, and Kitchen—and two real-world bimanual tasks, SinglePillDispense and SlipIntoBaggie.

Figure 1: Simulation benchmarks spanning precision assembly, planar pushing, gear insertion, and multimodal kitchen manipulation.

Controlled evidence for expert non-Markovianity

The strongest evidence comes from controlled comparisons in which the policy architecture, training protocol, evaluation procedure, and short context length are held fixed while the expert data are changed. The principal experiments use Diffusion Policies with $T_o=2$ observation frames and prediction horizon $T_p=15$, while sweeping execution horizons from one to fifteen actions.

For FurnitureSimOneLeg, the authors compare human demonstrations, demonstrations from a deterministic Markovian finite-state-machine expert, and demonstrations from a scripted expert augmented with non-Markovian behavior. The Markovian expert uses current and previous environment states to select waypoint-specific operational-space and gripper-control actions, making it Markovian relative to the learner’s two-frame input. The non-Markovian variant introduces hidden waypoint offsets, sticky state transitions, and latent-count alignment maneuvers. These mechanisms generate pauses, repeated alignment actions, and delayed transitions that are not identifiable from the current observation alone.

The result is a clear qualitative reversal. Human demonstrations produce the conventional inverted-U success-horizon curve. Markovian demonstrations produce an approximately monotonically decreasing curve whose optimum is $T_{\mathrm{exec}}^*=1$. Injecting non-Markovian behaviors into the scripted expert restores the inverted-U shape.

Figure 2: Human and non-Markovian scripted demonstrations induce intermediate-horizon optima, whereas Markovian demonstrations favor the most reactive execution horizon.

The same pattern appears in PushT-M, GearInsertion, and Kitchen. Policies trained on human data exhibit inverted-U curves, while policies trained on Markovian scripted demonstrations become monotonically decreasing and perform best at, or near, $T_{\mathrm{exec}}=1$.

Figure 3: Across PushT-M, GearInsertion, and Kitchen, changing expert Markovianity shifts the preferred execution horizon toward fully closed-loop control.

This comparison supports a causal interpretation within the experimental design: expert temporal structure, rather than the task alone, determines whether short execution horizons are viable. The implication is operationally important. A benchmark’s use of long action chunks cannot be interpreted as evidence that the underlying manipulation problem intrinsically requires open-loop control. It may instead indicate that the learner is under-contextualized relative to the demonstrator.

The observed failures are consistent with this account. In FurnitureSimOneLeg, short-context policies repeatedly align the table leg but fail to transition to grasping or insertion. In GearInsertion, the robot repeatedly aligns the gear with the shaft. In PushT, the policy switches between pushing modes or jitters until timeout. In Kitchen, it may approach one subtask and then abruptly switch to another locally plausible behavior. These are not simply large geometric errors; they are failures to select the temporally appropriate mode among multiple actions that are individually supported by the demonstrations.

Figure 4: Short-context, short-horizon policies fail through repeated alignment, mode switching, oscillation, and premature transitions.

Hidden-state aliasing and cycling

The paper’s conceptual explanation is hidden-state aliasing. Suppose the same or similar current observation occurs at different points in demonstrations, but the expert chooses different actions because its latent state differs. A short-context behavior-cloning policy must aggregate these action distributions. Even if it perfectly estimates the conditional distribution available from its input, that conditional distribution is ambiguous because the input omits the information that disambiguates the expert’s behavior.

Idle periods provide a simple instance. A human may pause while aligning, observing, or deciding whether a manipulation has succeeded. The resulting sequence of near-identical observations can encode an implicit counter or phase variable. A policy with only two frames cannot determine whether it should continue pausing or advance to the next subtask. It may consequently pause too long, advance too early, or repeatedly return to the same local behavior.

Figure 5: A sequence of visually similar idle states aliases the expert’s latent progress or timing state for a short-context learner.

The aggregation of multiple demonstrations can create cycles even when no individual demonstration contains a literal loop. At an aliased observation, one demonstration may support continuing toward one mode while another supports exiting toward a different mode. Replanning from the aliased state can cause the policy to alternate between these trajectories, producing an induced cycle.

Figure 6: Dataset-level aggregation can create cycles when locally valid transitions from different demonstrations are combined by a short-context imitator.

Long execution horizons suppress this failure probabilistically rather than resolving its cause. If replanning occurs every $T_{\mathrm{exec}}$ actions, an aliased state is encountered at a replanning boundary less frequently. An open-loop chunk can also carry the policy across a short cycle before another decision is made. Thus, action chunking acts as a commitment mechanism: it reduces the number of opportunities to make an inconsistent decision. This explains why long-horizon execution can improve performance even in synchronous simulation with zero inference latency.

The limitation of this mechanism is equally clear. Open-loop execution does not infer the hidden state; it merely reduces the frequency at which the policy must act under ambiguity. If an environmental perturbation occurs within the chunk, the policy cannot respond until the chunk terminates. Long execution horizons therefore exchange reduced decision ambiguity for reduced feedback responsiveness.

Compounding errors as a secondary explanation

The paper directly tests the alternative hypothesis that long execution horizons are principally beneficial because they reduce compounding errors. Two interventions are used on FurnitureSimOneLeg: increasing the number of demonstrations and applying HG-DAgger.

Dataset-scale experiments use 100, 200, 400, and 1,000 demonstrations from both Markovian and non-Markovian scripted experts. At the smallest scale, compounding errors substantially affect the curve: even the Markovian-expert policy prefers a nontrivial execution horizon of four actions. Above 200 demonstrations, however, additional data produce only modest shifts toward shorter horizons.

Figure 7: Compounding errors strongly affect the low-data regime but have a comparatively limited effect once moderate state coverage is available.

HG-DAgger adds 40 correction demonstrations per round for three rounds, separately for each execution horizon. It produces a slight shift toward shorter horizons for Markovian-expert policies and no noticeable shift for non-Markovian-expert policies.

Figure 8: HG-DAgger modestly improves the Markovian-expert curve but does not remove the short-context failures induced by non-Markovian demonstrations.

These results do not reject compounding errors. They show instead that their influence is conditional on data scale and expert structure. With only 100 demonstrations, insufficient state coverage can make even a Markovian imitation policy vulnerable to distribution shift. In the moderate-to-high-data regime emphasized by the paper, however, the effect is substantially weaker than the effect of changing expert Markovianity. HG-DAgger corrects missed insertions and basic manipulation errors but does not eliminate cyclic or idle behaviors when the corrective demonstrations remain non-Markovian.

The paper also reports exploratory noise-injection experiments. Moderate noise and a suitable fraction of clean trajectories can improve absolute performance, including at both low and high execution horizons, but the intervention does not clearly reshape the success-horizon curve toward shorter execution horizons. The authors appropriately qualify this result: noise injection can introduce additional multimodality, and multi-step targets collected after perturbed states create a state–action-sequence mismatch. Consequently, this ablation is not a clean test of the compounding-error hypothesis.

Long context as a reactive alternative

The paper’s principal remedy is to increase observation context rather than preserve behavioral coherence through open-loop commitment. If long execution horizons reduce the probability of replanning under hidden-state aliasing, long contexts can instead disambiguate the latent state while retaining frequent feedback.

On FurnitureSimOneLeg with 200 human demonstrations, increasing context from $T_o=2$ to $T_o=20$ progressively shifts the peak of the success-horizon curve toward shorter execution horizons. At context lengths of 16 and 20, the best policies use execution horizons of one or two actions.

Figure 9: Increasing visual context shifts the optimal execution horizon from an intermediate value toward reactive execution.

The effect becomes stronger with more data. With 1,000 non-Markovian scripted demonstrations, long-context policies using execution horizons below four actions achieve 93.2% success, compared with 90.6% for the best short-context policy using $T_o=2$ and $T_{\mathrm{exec}}$0.

Figure 10: At higher data scale, long-context reactive policies outperform short-context policies that rely on longer open-loop execution.

This result is central because it establishes not merely a reallocation of performance across execution horizons, but an absolute advantage for the reactive regime. The implication is that action-sequence prediction and reactive execution need not be treated as opposing design choices: a policy may predict a long sequence while executing only a very short prefix, provided its context is sufficient to maintain temporal coherence.

The same qualitative behavior is observed in PushT-D, GearInsertion, and Kitchen. With sufficiently long context, the success-horizon curves become nearly monotonically decreasing, making $T_{\mathrm{exec}}$1 optimal or near-optimal.

Figure 11: Long context consistently reduces or eliminates the performance benefit of long execution horizons across additional simulation tasks.

The real-world experiments extend the result beyond synchronous simulation. In SinglePillDispense, the robot must dispense exactly one pill and stop promptly; in SlipIntoBaggie, it must insert a paper slip into a nearly conformal bag without crumpling it. Increasing context from two to eight frames shifts the best execution horizon from eight to two actions and substantially improves success. SinglePillDispense reaches 90% at $T_{\mathrm{exec}}$2, $T_{\mathrm{exec}}$3, compared with 50% at $T_{\mathrm{exec}}$4, $T_{\mathrm{exec}}$5. SlipIntoBaggie reaches 85% under the same context and execution horizon, compared with 60% for the corresponding short-context policy.

Task	$T_{\mathrm{exec}}$6, best reported setting	$T_{\mathrm{exec}}$7, $T_{\mathrm{exec}}$8
SinglePillDispense	75% at $T_{\mathrm{exec}}$9	90%
SlipIntoBaggie	80% at $T_o=2$0	85%

The real-world implication is direct: reactivity is useful not only for dynamic environments but also for high-precision manipulation with uncertain contact dynamics, deformable objects, and termination conditions requiring immediate correction. In SinglePillDispense, feedback enables the robot to stop after one pill. In SlipIntoBaggie, it enables correction for paper or bag deformation. Inference latency is present in these experiments, yet longer context still permits shorter execution horizons, indicating that the result is not an artifact of idealized synchronous simulation.

Architecture, optimization, and empirical qualifications

Long-context learning is not achieved by merely increasing the temporal input dimension without qualification. For several tasks, the authors introduce a double-encoder cross-attention U-Net. A long-range encoder processes all frames, while a separately parameterized short-range encoder processes the most recent two frames. Short-range feature dropout prevents the model from ignoring long-range information. The architecture yields performance trends similar to a conventional cross-attention U-Net but produces smoother success-horizon curves and more stable long-context behavior.

The paper also identifies a training interaction that is important for evaluating reactive policies. Lower execution horizons generally favor earlier checkpoints, and longer context lengths often do so as well. In SinglePillDispense, selecting the latest checkpoint rather than an earlier one can reduce success by as much as 40 percentage points. This indicates that validation loss and closed-loop rollout success are not interchangeable model-selection criteria, particularly when the policy must operate on its own induced state distribution at high replanning frequency.

Several qualifications constrain the generality of the conclusions. First, the main comparisons are concentrated in a moderate-to-high-data regime, generally using at least 200 demonstrations. At 100 demonstrations, compounding errors can dominate sufficiently to make long execution beneficial even for Markovian experts. Second, Markovianity is defined relative to the observation and auxiliary information supplied to the policy. The environment observations are approximated as Markovian, and success filtering can theoretically induce residual non-Markovianity, although the scripted experts have success rates above 90%. Kitchen additionally receives a subtask-sequence label for the scripted-expert comparison, while human demonstrations do not.

Third, the context-length results are architecture- and representation-dependent. PushT-M is an explicit counterexample to uniformly beneficial context scaling: longer contexts substantially degrade absolute performance, which the authors attribute to visually sparse rendering and additional representation-level aliasing. The real-world evaluation uses only 20 trials per configuration and was conducted within constrained lighting conditions; the authors also note that color-jitter augmentation was unintentionally omitted. Finally, the study does not fully separate expert non-Markovianity from demonstration multimodality, temporal smoothness, architecture, checkpoint selection, and inference-time action interpolation. These factors may alter the magnitude of the success-horizon curve even when they do not change its qualitative explanation.

The paper therefore leaves a specific methodological question open: how should context length, dataset size, architecture, and checkpoint selection be jointly scaled to optimize reactive control without incurring excessive sample complexity or causal confusion? It also remains unresolved whether compressed or semantic memory representations can preserve the local visual information required for $T_o=2$1 control, rather than only task-level historical information.

Conclusion

The paper presents a controlled account of why long open-loop execution is effective in contemporary imitation learning. Across four simulated and two real-world tasks, it finds that short-context policies trained on non-Markovian demonstrations develop inverted-U-shaped success-horizon curves, whereas policies trained on Markovian demonstrations generally prefer fully reactive execution. Compounding errors remain relevant, especially with very small datasets, but exert a weaker influence than expert non-Markovianity in the principal regime studied.

The proposed interpretation is that action chunking suppresses hidden-state aliasing and cycling by reducing replanning opportunities. Increasing context addresses the same ambiguity directly while preserving feedback. The strongest empirical result is therefore not that long action sequences are unnecessary, but that long-context policies can execute those sequences reactively and achieve higher task performance. This reframes execution-horizon selection as a consequence of observability and training regime rather than as a fixed requirement of imitation-based robotic manipulation.