Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)

Multidimensional policy-parameter resonance analysis

Establish the resonance behavior of the primal-dual learning recursion for a multidimensional policy parameter, where the constraint gradient is not an eigenvector of the Lagrangian curvature, and quantify the error introduced by reducing the coupled system of dimension d+1 to a scalar curvature–sensitivity pair.

Background

The paper’s analytical results use a scalar tangent linear-quadratic reduction characterized by the curvature–sensitivity pair (a,c). This reduction is exact only for a one-dimensional policy parameter or when the constraint gradient is an eigenvector of the Lagrangian curvature.

For a general d-dimensional policy parameter, the resonance is generated by a coupled system of dimension d+1. The paper anticipates that the mechanism should persist in the mode most strongly coupled to the constraint, but does not establish this or control the error incurred by replacing the full curvature structure with the single scalar a.

References

Two extensions remain open. The first is a multidimensional policy parameter: the scalar pair (a, c) is exact only when d = 1 or when the constraint gradient is an eigenvector of the curvature, and otherwise the tongue comes from a coupled system of dimension d + 1. We expect the mechanism to persist on the mode most strongly coupled to the constraint, but reducing a d-dimensional curvature to the single number a carries an uncontrolled error; closing that gap is a spectral question about the pencil, not a new dynamical mechanism.

— Parametric Resonance in Primal-Dual Learning for Slowly Non-Stationary Constrained MDPs  (2610.23788 - Baelde, 16 Oct 2026) in Section 8, Conclusion

A separate, unresolved puzzle: Stooke et al. (2020, Figure 1) sweep the integral gain KI , our α, and observe the cost oscillations shrink and vanish as the gain grows, which read naively runs against Proposition 1 and Theorem 1. We have not run their sweep ourselves. A conjectured reconciliation: their perturbation has a spectrum fixed by the rate of policy improvement rather than swept, so as α grows ˜ω0 ∝ √α leaves that excitation band, predicting an amplitude non-monotone in KI , maximal where ˜ω0 crosses it. That prediction is testable on the same sweep, which we leave to future work.

— Parametric Resonance in Primal-Dual Learning for Slowly Non-Stationary Constrained MDPs  (2610.23788 - Baelde, 16 Oct 2026) in Section 8, Conclusion