Positive-semigroup argument under function approximation

Determine which parts of the positive-semigroup argument for variance-sensitive local concentration in quantile temporal-difference learning survive under function approximation.

Background

The local analysis relies on the QTD mean-field Jacobian being a nonsingular M-matrix whose discrete semigroup is positive and substochastic. This structure enables the variance–drift matching argument that removes an artificial inverse-density factor from the leading stochastic term. The paper does not resolve whether analogous structural properties remain available when tabular parameters are replaced by function approximators.

References

Several extensions remain open. It would be useful to obtain matching lower bounds for the global burn-in, to study asynchronous and Markovian sampling, and to determine which parts of the positive-semigroup argument survive under function approximation.

— A Finite Sample Analysis for Quantile Temporal Difference Learning in Distributional Reinforcement Learning  (2608.27313 - Cheng et al., 27 Aug 2026) in Section Conclusions