Quantile Optimal Treatment Regimes (OTRs)

• Quantile optimal treatment regimes are treatment rules that replace the conventional mean outcome objective with a targeted quantile objective, optimizing specific distributional characteristics.
• They employ diverse methods like pinball loss regression, dynamic Q-learning, and classification-based binary search to enhance decision-making in both single- and multi-stage settings.
• Advanced frameworks address challenges such as partial identification, minimax regret, and high-dimensional inference to improve robustness and precision in treatment policy learning.

Searching arXiv for primary and adjacent work on quantile optimal treatment regimes. {"query":"all:(\"Quantile-Optimal Treatment Regimes\" OR \"quantile optimal treatment regimes\" OR \"Interactive Q-learning for Probabilities and Quantiles\" OR \"Robust regression for optimal individualized treatment rules\")","max_results":10,"sort_by":"relevance"} I found several relevant arXiv records, including core work on robust regression for individualized treatment rules, dynamic quantile-oriented Q-learning, and recent sequential-classification and minimax-regret formulations for quantile-based treatment choice. Searching specifically for Wang et al.'s quantile OTR formulation and closely related policy-learning work. {"query":"all:(Wang quantile optimal treatment regimes)","max_results":10,"sort_by":"relevance"} Quantile optimal treatment regimes (OTRs) are treatment rules that replace the usual mean-outcome objective with a quantile-based objective, so that treatment assignment is chosen to improve a specified part of the outcome distribution, such as the lower tail, the median, or an upper quantile. In the literature surveyed here, this idea appears in several non-equivalent forms: maximizing a conditional quantile $Q_\tau(X,a)$ and choosing treatment pointwise by covariates; maximizing the quantile of the regime-induced counterfactual outcome $Y(d)$; optimizing a dynamic regime through the quantile of the induced multi-stage outcome distribution; and learning policies from the conditional quantile of the individual treatment effect $Y_1-Y_0$. The resulting field is therefore unified by a distributional objective, but not by a single formal estimand or a single estimation strategy (Xiao et al., 2016, Linn et al., 2014, Xia et al., 15 Jul 2025, Cui et al., 2023).

1. Formal targets and non-equivalent notions of quantile optimality

A single-stage individualized treatment rule may be written as $g:\mathcal X\to\mathcal A$ or $d:\mathcal X\to\{0,1\}$, with observed data $(X,A,Y)$, binary treatment, and outcomes coded so that larger values are better. One formalization defines the $\tau$-th conditional quantile of $Y$ given $(X,A)$ by

$$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$

then introduces the quantile-based value

$Y(d)$0

and defines the $Y(d)$1-quantile-optimal treatment rule as

$Y(d)$2

A different single-stage formulation defines the counterfactual regime outcome

$Y(d)$3

the regime survival function

$Y(d)$4

and the target

$Y(d)$5

In two-stage dynamic regimes, the object becomes

$Y(d)$6

with the optimal regime maximizing the quantile of the counterfactual outcome distribution under a sequential rule $Y(d)$7. A further variant, developed in policy learning with distributional welfare, uses the conditional quantile of the individual treatment effect: $Y(d)$8 These objectives are related but not interchangeable (Xiao et al., 2016, Linn et al., 2014, Xia et al., 15 Jul 2025, Cui et al., 2023).

Framework	Quantile target	Setting
Robust regression OTR	$Y(d)$9 and $Y_1-Y_0$0	Single-stage, binary treatment
Sequential classification learning	$Y_1-Y_0$1 via $Y_1-Y_0$2	Single-stage, with two-stage extension
Interactive Q-learning for quantiles	$Y_1-Y_0$3	Two-stage dynamic regime
Distributional welfare policy learning	$Y_1-Y_0$4	Binary treatment policy learning

The distinction between these targets is consequential. Quantiles of outcomes under treatment and control, quantiles of the regime-induced outcome distribution, and quantiles of individual gains $Y_1-Y_0$5 answer different policy questions. This suggests that “quantile OTR” is best treated as a family of distribution-sensitive treatment objectives rather than a single canonical criterion (Cui et al., 2023).

2. Single-stage robust regression and pinball-loss regimes

A central single-stage formulation rewrites the conditional mean through the centered treatment term: $Y_1-Y_0$6 where $Y_1-Y_0$7, $Y_1-Y_0$8 is a baseline effect, and $Y_1-Y_0$9 is the treatment contrast. The corresponding decision rule is determined only by the sign of the contrast,

$g:\mathcal X\to\mathcal A$0

The robust-regression working model is

$g:\mathcal X\to\mathcal A$1

with estimator

$g:\mathcal X\to\mathcal A$2

and induced regime

$g:\mathcal X\to\mathcal A$3

When $g:\mathcal X\to\mathcal A$4 is the pinball or check loss,

$g:\mathcal X\to\mathcal A$5

the loss targets conditional quantiles rather than conditional means (Xiao et al., 2016).

Under the additive model

$g:\mathcal X\to\mathcal A$6

the conditional quantile is

$g:\mathcal X\to\mathcal A$7

so the treatment comparison is governed by the same contrast term at every $g:\mathcal X\to\mathcal A$8. Consequently,

$g:\mathcal X\to\mathcal A$9

In that exact case, pinball-loss learning yields the same treatment boundary as the mean-optimal rule, while retaining robustness to skewed, heterogeneous, heavy-tailed errors and outliers (Xiao et al., 2016).

When $d:\mathcal X\to\{0,1\}$0 fails, the quantile interpretation is no longer exact in general. The paper then assumes a conditional quantile working model

$d:\mathcal X\to\{0,1\}$1

defines the specification error

$d:\mathcal X\to\{0,1\}$2

and shows in Theorem 2 that the population pinball minimizer satisfies

$d:\mathcal X\to\{0,1\}$3

Thus the fitted pinball model is a weighted least-squares approximation to the true conditional quantile function, and the induced rule approximately maximizes the $d:\mathcal X\to\{0,1\}$4-th conditional quantile. The same framework establishes consistency and asymptotic normality for the contrast estimator under stated regularity conditions, with the caveat that the core consistency result relies on the stronger condition $d:\mathcal X\to\{0,1\}$5, not merely a mean-zero error restriction (Xiao et al., 2016).

3. Dynamic quantile regimes and Interactive Q-learning

In the dynamic setting, the observed data are

$d:\mathcal X\to\{0,1\}$6

with histories

$d:\mathcal X\to\{0,1\}$7

binary treatments at each stage, and regime $d:\mathcal X\to\{0,1\}$8. The dynamic quantile objective is defined through the distribution of the counterfactual outcome $d:\mathcal X\to\{0,1\}$9: $(X,A,Y)$0 and a quantile-optimal regime maximizes $(X,A,Y)$1 over all regimes. The framework also considers threshold objectives such as

$(X,A,Y)$2

This directly extends optimal treatment regime methodology from mean optimization to distributional criteria (Linn et al., 2014).

The underlying outcome model at stage 2 is

$(X,A,Y)$3

with $(X,A,Y)$4 independent of $(X,A,Y)$5. For threshold optimization, the optimal second-stage rule is

$(X,A,Y)$6

because $(X,A,Y)$7. The first-stage rule is more intricate: it depends on the full induced future distribution through

$(X,A,Y)$8

where $(X,A,Y)$9 is the joint conditional distribution of $\tau$0 and $\tau$1 given first-stage history and treatment. The threshold-optimal first-stage rule is

$\tau$2

This is the basis of TIQ-learning (Linn et al., 2014).

Quantile Interactive Q-learning (QIQ-learning) solves the harder quantile problem by searching over threshold-indexed TIQ rules. Defining

$\tau$3

the method introduces

$\tau$4

and, under continuity and strict monotonicity of $\tau$5, uses

$\tau$6

The resulting procedure estimates $\tau$7, $\tau$8, $\tau$9, and $Y$0, computes $Y$1, and recovers a regime whose attained quantile is consistent for the target. Theorem 2 establishes consistency of TIQ-learning for $Y$2, and Theorem 3 establishes consistency of QIQ-learning for the target quantile (Linn et al., 2014).

This framework also clarifies why standard mean-based Q-learning is insufficient for quantile objectives. Quantiles are nonlinear functionals, so Bellman-style mean recursion does not directly apply. The paper further shows that replacing $Y$3 by $Y$4 in binary Q-learning does not generally recover a genuine threshold- or quantile-optimal rule; under its generative model, the resulting estimand reduces to the same object as mean-based Q-learning and can fail to vary with $Y$5 (Linn et al., 2014).

4. Sequential classification, binary search, and discrete outcomes

A recent single-stage development reformulates quantile OTR estimation as a sequence of weighted classification problems. The starting point is

$Y$6

with

$Y$7

For each fixed threshold $Y$8, the inner problem is

$Y$9

and the optimal quantile value $(X,A)$0 is the largest threshold such that

$(X,A)$1

The estimation strategy therefore uses an outer binary search over $(X,A)$2 and an inner classification problem for $(X,A)$3 (Xia et al., 15 Jul 2025).

The crucial representation is doubly robust: $(X,A)$4 where

$(X,A)$5

and

$(X,A)$6

This yields the weighted classification equivalence

$(X,A)$7

Using a decision function $(X,A)$8 with $(X,A)$9 and hinge loss $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$0, the estimator becomes

$$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$1

with $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$2 taken from an RKHS and either a linear kernel or the Gaussian kernel

$$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$3

The estimated survival under the estimated rule is evaluated through the smoothed plug-in estimator

$$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$4

The resulting algorithm tunes $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$5 and, for the Gaussian kernel, $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$6 by $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$7-fold CV, and recommends $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$8, $$Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\},$$9, and $Y(d)$00 (Xia et al., 15 Jul 2025).

A major contribution concerns discrete outcomes. For discrete $Y(d)$01, the survival function $Y(d)$02 is piecewise constant, the quantile is stepwise, direct empirical quantile maximization can be inconsistent, and tied quantile-optimal regimes raise an “ineffectiveness” issue. The proposed remedy linearly interpolates the survival function between adjacent support points $Y(d)$03: $Y(d)$04 and defines the smoothed quantile

$Y(d)$05

The key theoretical inclusion is

$Y(d)$06

so any regime maximizing the smoothed quantile also maximizes the original quantile. Under Assumption 9, which requires the sign of

$Y(d)$07

to remain invariant across $Y(d)$08, Corollary 1 further gives

$Y(d)$09

Theorem 3 establishes quantile-value consistency in the continuous case, and Theorem 4 proves that for discrete outcomes

$Y(d)$10

Simulation results favor QIQ-learning in a correctly specified linear setting, but SCL-Gaussian performs best in nonlinear continuous and discrete settings, and the ACTG175 analysis reports the highest estimated quantile values for SCL-Gaussian at $Y(d)$11 (Xia et al., 15 Jul 2025).

5. Identification, partial identification, and minimax regret

One important branch of the literature argues that individualized quantile treatment choice should be based on the conditional quantile of the individual treatment effect,

$Y(d)$12

rather than on the difference of conditional quantiles

$Y(d)$13

The reason is that the person at quantile $Y(d)$14 under treatment need not be the same person at quantile $Y(d)$15 under control, so

$Y(d)$16

in general. This reframes quantile OTRs as policies that maximize

$Y(d)$17

with the unconstrained first-best rule

$Y(d)$18

At $Y(d)$19, the median criterion admits a majority-benefit interpretation; for continuous $Y(d)$20,

$Y(d)$21

This makes the median QoTE rule a majority-vote rule in the sense formalized in Theorem 1 (Cui et al., 2023).

The difficulty is identification. Even with $Y(d)$22, the distribution of $Y(d)$23 depends on the unobserved joint distribution of $Y(d)$24, so the QoTE is generally only partially identified. The baseline bounds are Makarov-type bounds: $Y(d)$25 with

$Y(d)$26

and

$Y(d)$27

Stronger assumptions such as positive dependence, joint conditional independence, deconvolution, Roy-type models, rank invariance, and symmetry can tighten or point identify the target. Under rectangularity of the identified set, minimax regret simplifies to maximization of

$Y(d)$28

where

$Y(d)$29

The optimal stochastic robust policy is

$Y(d)$30

and the optimal deterministic robust policy chooses the sign with smaller worst-case regret in the ambiguous region $Y(d)$31 (Cui et al., 2023).

A distinct but closely related decision-theoretic analysis studies finite-sample minimax regret when the object of interest is a quantile of the realized outcome distribution under a treatment rule,

$Y(d)$32

rather than a conditional quantile $Y(d)$33 or a QoTE. In the designs with fixed treated and untreated sample sizes or with random assignment, Proposition 1 shows that any treatment rule is minimax regret optimal, with

$Y(d)$34

for all rules under the stated quantile-selection conditions. In the “testing an innovation” design, if the known untreated quantile equals $Y(d)$35, any rule is minimax regret; if it exceeds $Y(d)$36, never treating is the unique minimax rule; and if it is below $Y(d)$37, always treating is a minimax rule. These results persist under several restrictions on nature, including Bernoulli outcomes in designs (i) and (ii). The paper therefore functions as a cautionary adjacent result: under a robust finite-sample minimax-regret criterion, quantile-based treatment choice can become non-discriminating or collapse to trivial no-data rules (Guggenberger et al., 6 Jan 2026).

6. High-dimensional contrasts, inference, and recurring distinctions

High-dimensional individualized quantile treatment effect inference provides another route into quantile-sensitive treatment assignment. In a two-group setup with

$Y(d)$38

the target individualized contrast for a new subject $Y(d)$39 is

$Y(d)$40

which corresponds to

$Y(d)$41

under the treatment-group interpretation. The paper does not formulate a regime value function, but it explicitly notes the one-sided testing problem

$Y(d)$42

so a natural quantile-targeted treatment rule is

$Y(d)$43

This is a direct implication of the treatment-comparison framework rather than a formal OTR theorem (Sun et al., 24 Mar 2025).

The methodology debiases the linear functional directly. For each treatment group, it constructs a projection direction $Y(d)$44 by constrained optimization, with the novel variance-enhancement constraint

$Y(d)$45

and then defines the debiased estimator

$Y(d)$46

where $Y(d)$47. This yields the IQTE estimator

$Y(d)$48

The paper proves pointwise asymptotic normality,

$Y(d)$49

constructs confidence intervals and one-sided tests, establishes weak convergence over $Y(d)$50, and derives minimax-optimal rates for expected CI length and testing detection boundary. This makes it a rigorous inferential module for quantile-sensitive treatment comparison in high-dimensional settings, even though it is not itself an OTR learning paper (Sun et al., 24 Mar 2025).

Across these strands, several recurring distinctions structure the subject. First, mean-optimal and quantile-optimal rules need not coincide: treatments can share the same conditional mean and differ in lower or upper quantiles, and simulations in both the robust-regression and dynamic-learning literatures show that rules tuned to $Y(d)$51, $Y(d)$52, or other targets can prioritize materially different parts of the outcome distribution (Xiao et al., 2016, Linn et al., 2014). Second, quantile optimization is not merely “robust mean regression”: pinball-loss OTRs can be exact or approximate quantile procedures depending on the model and independence assumptions, and dynamic quantile regimes require distributional recursion rather than ordinary mean-based Bellman updates (Xiao et al., 2016, Linn et al., 2014). Third, direct quantile value search can be computationally unstable or inconsistent for discrete outcomes, which motivates classification-based reformulations and smoothing constructions (Xia et al., 15 Jul 2025). Fourth, quantile criteria can be fundamentally harder than mean criteria under worst-case finite-sample decision theory, because quantiles are discontinuous functionals of the distribution and can make minimax regret largely uninformative (Guggenberger et al., 6 Jan 2026).

Taken together, the literature presents quantile OTRs as a family of regime-learning problems in which the target is a chosen feature of the outcome distribution rather than the expectation. The main lines of development are robust single-stage contrast modeling through pinball loss, dynamic distributional learning through TIQ- and QIQ-learning, sequential classification with binary search and doubly robust survival representations, partial-identification and minimax-regret policies based on quantiles of individual gains, and high-dimensional inference for individualized conditional quantile contrasts. This suggests that the defining issue in quantile OTRs is not only which algorithm is used, but also which quantile object is being optimized and under what causal, structural, and decision-theoretic assumptions (Xiao et al., 2016, Linn et al., 2014, Xia et al., 15 Jul 2025, Cui et al., 2023, Sun et al., 24 Mar 2025, Guggenberger et al., 6 Jan 2026).