Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure

Abstract: A central challenge in deploying learned robot policies is inference-time behavior steering: redirecting a policy at test time to satisfy user preferences not anticipated during training, without retraining. Existing methods fail in two modes: end-to-end methods require fine-tuning or expert-level guidance, while neuro-symbolic methods rely on predefined symbols whose edits can result in logically reasonable but physically infeasible plans. To address this challenge, we propose ReStruct, which builds upon a neural automaton policy that decomposes a visuomotor policy into a high-level state-machine skeleton capturing task structure and a low-level continuous controller represented as a residual policy. Specifically, ReStruct adopts the automaton to represent the preference and incorporates it into the skeleton through a synchronous product, thereby reconfiguring the task structure. With the controller kept frozen, the action priors provided by the skeleton are updated accordingly to enable physically-aware control under a modified task structure. Extensive experiments from simulation and real-world show that ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints, and after steering surpasses existing methods, exceeding VLA models in both task success and preference-following by up to 25%.

• The paper introduces ReStruct, a framework that reconfigures a symbolic task structure to steer robot behavior without retraining.
• It translates natural language preferences into LTLf formulas and DFAs, then replays demonstration trajectories to update action priors for physical feasibility.
• Experimental results show up to a 25% improvement in task success and preference compliance across diverse manipulation tasks.

Physically-Aware Inference-Time Robot Behavior Steering via Task Structure Reconfiguration

Problem Formulation and Motivation

The challenge addressed by this work is inference-time robot behavior steering: adapting pretrained robot policies to new user preferences at test time without retraining or requiring expert-level guidance. Traditional approaches fall short in two primary modes. End-to-end approaches—including vision-language-action (VLA) models and diffusion-based policies—necessitate either retraining or constraint injection, which restricts their adaptability to preferences expressible as differentiable signals or demands domain expertise. Neuro-symbolic methods, which allow semantic-level preference specification, suffer from an inability to guarantee that symbolic modifications are propagable to the low-level controller, frequently resulting in plans that are logically consistent but physically infeasible.

ReStruct Framework: Physically-Aware Policy Steering

The paper introduces ReStruct, a framework for zero-shot robot policy adaptation by reconfiguring the symbolic task structure of a pretrained policy. ReStruct leverages the Emergent Neural Automaton Policy (ENAP), which decouples the visuomotor policy into a high-level probabilistic automaton (PMM) skeleton and a low-level residual controller. Steering proceeds without retraining the controller, enabling practical deployment in scenarios requiring runtime adaptability.

Core methodology:

• User preferences (defined in natural language) are compiled into Linear Temporal Logic over Finite Traces (LTLf) formulas.
• These formulas are converted into Deterministic Finite Automata (DFA).
• The DFA is synchronously product-composed with the base skeleton, yielding a reconfigured skeleton that generates only preference-adhering plans.
• Crucially, ReStruct recomputes edge-specific action priors by replaying demonstration trajectories through the refined skeleton, discarding those that violate the new specification.
• The frozen controller consumes these updated priors, ensuring that symbolic steering is translated into physically realizable actions.

This approach provides interpretability and symbolic verifiability at both the skeleton and controller levels. Moreover, it maintains policy data efficiency and cross-task adaptability, reusing experience from diverse demonstrations to synthesize target-task behaviors.

Technical Contributions

Skeleton Reconfiguration and Preference Grounding

The high-level abstraction z in the Bayesian factorization represents task structure as a PMM skeleton. ReStruct constructs a steered skeleton by transforming the user preference into a DFA and taking a synchronous product with the base skeleton. When preference expressivity exceeds the granularity of the clustering, a Vision-LLM (VLM) is employed to refine clusters prior to DFA compilation.

Action Prior Bridge and Replay

Action priors along product skeleton edges are reconstructed by trajectory replay. Representative trajectories from the demonstration set are accepted only if they conform to the preference-induced DFA. Their actions populate the edge priors, serving as a candidate pool for the controller. This procedure ensures that behavioral steering is physically grounded, avoiding the failure modes of previous neuro-symbolic methods that utilized priors anchored to preference-incompatible demonstrations.

Controller Freezing and Robust Prior Consumption

The low-level controller is kept frozen, consuming the updated action priors through a soft-attention mechanism and residual adaptation. The controller architecture integrates a prototype-based prior selection with a residual MLP correction—clamped to the feasible action space—trained via behavior cloning. This design counters the brittleness seen in earlier ENAP variants, which exposed the controller to only fixed priors during training.

Experimental Results

The paper reports extensive evaluation across simulated and real-world manipulation tasks, including StackCube, PegInsert, SeqPushT, ConeCover, CanSorting, PassTeaPot, and both unconstrained and causal skill sequences.

Key quantitative findings:

• ReStruct consistently achieves superior task success (SR) and preference adherence (SRP), often exceeding prior baselines by margins up to 25%.
• Comparative analyses against transformer, GMM, diffusion policy, and VLA models demonstrate ReStruct’s ability to recover ordered sequences from unordered demonstration data and to steer compositional out-of-domain behaviors.
• The statefulness imbued by the PMM skeleton yields greater decisiveness. Steering-induced reductions in episode lengths and improved multimodal robustness are observed, particularly on tasks with hybrid dynamics.
• Ablation studies confirm that both action prior replay and the attention-based controller are essential to maintaining performance and preference adherence; removing these components degrades both SR and SRP.

Theoretical and Practical Implications

The methodology advances physically-aware preference grounding, allowing symbolic steering to propagate down to controller-executable behaviors. This functional decoupling is validated by analysis showing that preference satisfaction and raw task completion can be independently modulated by the quality of PMM priors and policy training scale, respectively.

From a practical standpoint, ReStruct equips robots with runtime reconfigurability, facilitating zero-shot adaptation to new instructions and constraints in deployment settings. The symbolic skeleton provides interpretable runtime memory and historical commitment, markedly enhancing mode decisiveness and minimizing runtime uncertainty.

Limitations and Future Directions

Current limitations include:

• The reliance on ENAP means generalizing ReStruct to other policy classes requires distillation techniques, which may incur performance gaps.
• Success and preference adherence are dependent on the quality of VLM-generated LTL formulas, which may be improved with iterative refinement.
• The scope of steerable behaviors is bounded; scaling to richer out-of-domain regimes or eliciting novel skills remains unaddressed.

Future developments could explore scalable distillation across policy classes, more expressive symbolic alphabets generated by advanced LLMs, and broader compositional generalization regimes in robot control.

Conclusion

ReStruct delivers a physically-aware, inference-time robot behavior steering paradigm that reconfigures task structure to enforce user preferences without retraining. By jointly leveraging symbolic automata, trajectory replay for action prior grounding, and robust controller adaptation, ReStruct maintains high task success and preference compliance. The approach eliminates the structural limitations of previous neuro-symbolic and end-to-end methods, providing interpretable, verifiable, and physically realizable runtime steering and opening avenues for human-centric robot behavior customization (2606.26588).