Necessity of the Exponential Dependence on the Distance Parameter

Determine whether the exponential dependence on the distance parameter D in the sample complexity for learning transferable expected-hitting-time ranking supermartingale certificates is necessary, or whether it is merely an artifact of the worst-case bound in Theorem~\ref{thm:finite_hitting_time}.

Background

The paper proves that, under a known lower bound p_min on nonzero transition probabilities, an expected-hitting-time ranking supermartingale certificate can be learned from sampling access. The resulting sample complexity is polynomial in 1/alpha, where alpha=(p_min/2)^D/(2D), and therefore exponential in D, the maximum shortest-path distance to the target under the fixed policy. Because D is determined by the unknown transition-kernel support, the learner may have to use the worst-case bound D≤|S|−1.

The authors note that this exponential dependence may be overly conservative because it arises from the worst-case upper bound on expected hitting times, and their experiments suggest that the bound is loose in practice. The unresolved problem is to establish whether exponential dependence on D is fundamentally required or can be improved through a sharper analysis or a different certification-learning method.