Optimal dimension and horizon dependence for adversarial linear CMDPs

Establish optimal dependence on the feature dimension d and horizon length H for computationally efficient adversarial linear constrained Markov decision processes, improving the dependence achieved by the presented algorithm toward the near-optimal rate known for adversarial linear Markov decision processes.

Background

The paper establishes near-optimal dependence on the number of episodes K for both regret and cumulative constraint violation in adversarial linear constrained Markov decision processes. However, the authors note that the resulting dependence on the feature dimension d and horizon H is not optimal. They compare their bounds with the best-known guarantees for adversarial linear, unconstrained MDPs and identify improvement toward a near-optimal d√(H3K) rate as an unresolved objective for the constrained setting.

The open problem concerns the statistical dependence on d and H while retaining the paper’s computational efficiency, adversarial losses and constraints, unknown transitions, and lack of a Slater-condition assumption.

References

However, even in the unconstrained setting, no algorithm for adversarial linear MDPs has been proposed that achieves near-optimal guarantees, i.e., $(d\sqrt{H3 K})$~\citep{he2023nearly}. Therefore, we leave achieving optimal dependence on $d$ and $H$ as future work in our setting.

— Rate-Optimal Algorithm for Adversarial Linear CMDPs  (2610.00927 - Yu et al., 1 Oct 2026) in Section 3, paragraph following Theorem 1 (Main Result)

However, the main challenge arises in deriving eq:ingredient 1 modified. For comparison, to obtain eq:ingredient 1, we leverage the value difference lemma. On the other hand, when $[\cdot]+$ is imposed, such a strategy is no longer applicable, since $[\cdot]+$ destroys the linearity of the left-hand side. Therefore, it is nontrivial to obtain eq:ingredient 1 modified, which is a modified version of eq:ingredient 1 designed to ensure hard constraint violation. We leave this direction as future work.

— Rate-Optimal Algorithm for Adversarial Linear CMDPs  (2610.00927 - Yu et al., 1 Oct 2026) in Appendix, Section 'Additional Discussions', paragraph 'Limitations and Future Work'