Principled susceptibility proxies for continuous-control state selection

Develop principled susceptibility proxies for continuous control, such as measures based on representation sensitivity or local value gradients, to enable reliable sparse state selection for trajectory-level intervention.

Background

The paper investigates whether state-selective intervention can extend from discrete-action federated reinforcement learning to continuous-control settings using LunarLanderContinuous-v2. For discrete policies, normalized categorical entropy provides an inexpensive uncertainty signal for selecting intervention states. However, for Gaussian policies, state-independent variance yields entropy that is constant across states, while state-dependent variance can remain high throughout the state space and cause the intervention gate to activate on nearly every visited state.

The exploratory continuous-control experiments therefore use behavioral steering based on perturbing action means toward saturated control corners. Fixed perturbation scaling does not consistently degrade performance, and adaptive scaling succeeds under some transition dynamics but fails under others. The authors explicitly identify the development of principled susceptibility measures—potentially based on representation sensitivity or local value gradients—as unresolved work needed to obtain reliable sparse state selection in continuous control.

References

Developing principled susceptibility proxies for continuous control—such as measures based on representation sensitivity or local value gradients—remains an open direction.

— When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learning  (2610.11523 - Zheng et al., 8 Oct 2026) in Appendix E, Section E.2, Discussion

CheckVLA establishes that actionability depends on available repairs, but leaves open how intervention value changes across the trajectory and whether acting remains preferable once false alarms and intervention costs are included [5].

— RESCUE: Is This Alarm Worth Acting On? Counterfactual Evaluation of VLA Runtime Monitors  (2610.13516 - Ahmedt-Aristizabal et al., 9 Oct 2026) in Section I, Introduction, p. 1