--- title: Quantile Optimal Treatment Regimes (OTRs) url: https://www.emergentmind.com/topics/quantile-optimal-treatment-regimes-otrs type: topic --- # Quantile Optimal Treatment Regimes (OTRs) Searching arXiv for primary and adjacent work on quantile optimal treatment regimes. {"query":"all:(\"Quantile-Optimal Treatment Regimes\" OR \"quantile optimal treatment regimes\" OR \"Interactive Q-learning for Probabilities and Quantiles\" OR \"Robust regression for optimal individualized treatment rules\")","max_results":10,"sort_by":"relevance"} I found several relevant arXiv records, including core work on robust regression for individualized treatment rules, dynamic quantile-oriented Q-learning, and recent sequential-classification and minimax-regret formulations for quantile-based treatment choice. Searching specifically for Wang et al.'s quantile OTR formulation and closely related policy-learning work. {"query":"all:(Wang quantile optimal treatment regimes)","max_results":10,"sort_by":"relevance"} Quantile optimal treatment regimes (OTRs) are treatment rules that replace the usual mean-outcome objective with a quantile-based objective, so that treatment assignment is chosen to improve a specified part of the outcome distribution, such as the lower tail, the median, or an upper quantile. In the literature surveyed here, this idea appears in several non-equivalent forms: maximizing a conditional quantile \(Q_\tau(X,a)\) and choosing treatment pointwise by covariates; maximizing the quantile of the regime-induced counterfactual outcome \(Y(d)\); optimizing a dynamic regime through the quantile of the induced multi-stage outcome distribution; and learning policies from the conditional quantile of the individual treatment effect \(Y_1-Y_0\). The resulting field is therefore unified by a distributional objective, but not by a single formal estimand or a single estimation strategy [1604.03648] [1407.3414] [2507.11255] [2311.15878]. ## 1. Formal targets and non-equivalent notions of quantile optimality A single-stage individualized treatment rule may be written as \(g:\mathcal X\to\mathcal A\) or \(d:\mathcal X\to\{0,1\}\), with observed data \((X,A,Y)\), binary treatment, and outcomes coded so that larger values are better. One formalization defines the \(\tau\)-th conditional quantile of \(Y\) given \((X,A)\) by \[ Q_\tau(X,A)\triangleq \inf\{y: F_{Y\mid X,A}(y)\ge \tau\}, \] then introduces the quantile-based value \[ V_{\tau-q}(g)=E_X\big[Q_\tau\{X,g(X)\}\big], \] and defines the \(\tau\)-quantile-optimal treatment rule as \[ g_\tau^{\mathrm{opt}}(X)= \arg\max_{a\in\mathcal A} Q_\tau(X,a), \qquad g_\tau^{\mathrm{opt}}=\arg\max_g V_{\tau-q}(g). \] A different single-stage formulation defines the counterfactual regime outcome \[ Y(d)=Y(0)I\{d(\boldsymbol X)=0\}+Y(1)I\{d(\boldsymbol X)=1\}, \] the regime survival function \[ S(q,d)=P\{Y(d)>q\}, \] and the target \[ d^*=\argmax_d Q_\tau\{Y(d)\}, \qquad Q_\tau\{Y(d)\}=\sup\{q:S(q,d)>1-\tau\}. \] In two-stage dynamic regimes, the object becomes \[ q^\pi(\tau)=\inf\{y:\Pr^\pi(Y\le y)\ge \tau\}, \] with the optimal regime maximizing the quantile of the counterfactual outcome distribution under a sequential rule \(\pi=(\pi_1,\pi_2)\). A further variant, developed in policy learning with distributional welfare, uses the conditional quantile of the individual treatment effect: \[ Q_\tau(Y_1-Y_0\mid X), \qquad \delta_\tau^* \in \arg\max_{\delta\in\mathcal D} E[\delta(X)Q_\tau(Y_1-Y_0\mid X)]. \] These objectives are related but not interchangeable [1604.03648] [1407.3414] [2507.11255] [2311.15878]. | Framework | Quantile target | Setting | |---|---|---| | Robust regression OTR | \(Q_\tau(X,a)\) and \(V_{\tau-q}(g)\) | Single-stage, binary treatment | | Sequential classification learning | \(Q_\tau\{Y(d)\}\) via \(S(q,d)\) | Single-stage, with two-stage extension | | Interactive Q-learning for quantiles | \(q^\pi(\tau)\) | Two-stage dynamic regime | | Distributional welfare policy learning | \(Q_\tau(Y_1-Y_0\mid X)\) | Binary treatment policy learning | The distinction between these targets is consequential. Quantiles of outcomes under treatment and control, quantiles of the regime-induced outcome distribution, and quantiles of individual gains \(Y_1-Y_0\) answer different policy questions. This suggests that “quantile OTR” is best treated as a family of distribution-sensitive treatment objectives rather than a single canonical criterion [2311.15878]. ## 2. Single-stage robust regression and pinball-loss regimes A central single-stage formulation rewrites the conditional mean through the centered treatment term: \[ E(Y\mid X,A)=\varphi_0(X)+\{A-\pi(X)\}C_0(X), \] where \(\pi(X)=P(A=1\mid X)\), \(\varphi_0(X)\) is a baseline effect, and \(C_0(X)\) is the treatment contrast. The corresponding decision rule is determined only by the sign of the contrast, \[ g_\mu^{\mathrm{opt}}(X)=\mathbf 1\{C_0(X)>0\}. \] The robust-regression working model is \[ Y_i=\varphi_0(X_i)+\{A_i-\pi(X_i)\}C(X_i;\beta_0)+\epsilon_i, \] with estimator \[ (\hat\gamma_M^R,\hat\beta_M^R) = \arg\min_{\gamma,\beta} \frac1n\sum_{i=1}^n M\!\left[ Y_i-\varphi(X_i;\gamma)-\{A_i-\pi(X_i)\}C(X_i;\beta) \right], \] and induced regime \[ \hat g_M^R(X)=\mathbf 1\{C(X;\hat\beta_M^R)>0\}. \] When \(M\) is the pinball or check loss, \[ \rho_\tau(u)= \begin{cases} (\tau-1)u, & u<0,\\ \tau u, & u\ge 0, \end{cases} \] the loss targets conditional quantiles rather than conditional means [1604.03648]. Under the additive model \[ Y=\varphi_0(X)+\{A-\pi(X)\}C(X;\beta_0)+\epsilon, \qquad \epsilon\perp A\mid X, \] the conditional quantile is \[ Q_\tau(X,A)= \varphi_0(X)+\{A-\pi(X)\}C(X;\beta_0)+F_\epsilon^{-1}(X;\tau), \] so the treatment comparison is governed by the same contrast term at every \(\tau\). Consequently, \[ g_\mu^{\mathrm{opt}} = g_\tau^{\mathrm{opt}} = \mathbf 1\{C(X;\beta_0)>0\}. \] In that exact case, pinball-loss learning yields the same treatment boundary as the mean-optimal rule, while retaining robustness to skewed, heterogeneous, heavy-tailed errors and outliers [1604.03648]. When \(\epsilon\perp A\mid X\) fails, the quantile interpretation is no longer exact in general. The paper then assumes a conditional quantile working model \[ Q_\tau(X,A)=\varphi_0(X)+\{A-\pi(X)\}C(X;\beta_0(\tau)), \] defines the specification error \[ \Delta_\tau(X,A;\beta,\gamma)=\hat Q(X,A)-Q_\tau(X,A), \] and shows in Theorem 2 that the population pinball minimizer satisfies \[ (\beta(\tau),\gamma(\tau)) = \arg\min_{\beta,\gamma} E\!\left[ w_\tau(X,A;\beta,\gamma)\, \Delta_\tau^2(X,A;\beta,\gamma) \right]. \] Thus the fitted pinball model is a weighted least-squares approximation to the true conditional quantile function, and the induced rule approximately maximizes the \(\tau\)-th conditional quantile. The same framework establishes consistency and asymptotic normality for the contrast estimator under stated regularity conditions, with the caveat that the core consistency result relies on the stronger condition \(\epsilon\perp A\mid X\), not merely a mean-zero error restriction [1604.03648]. ## 3. Dynamic quantile regimes and Interactive Q-learning In the dynamic setting, the observed data are \[ \mathcal D=\{(\bX_{1i},A_{1i},\bX_{2i},A_{2i},Y_i)\}_{i=1}^n, \] with histories \[ \bH_1=\bX_1,\qquad \bH_2=(\bH_1,A_1,\bX_2), \] binary treatments at each stage, and regime \(\bpi=(\pi_1,\pi_2)\). The dynamic quantile objective is defined through the distribution of the counterfactual outcome \(Y^*(\bpi)\): \[ q^\pi(\tau)=\inf\{y:\Pr^\pi(Y\le y)\ge \tau\}, \] and a quantile-optimal regime maximizes \(q^\pi(\tau)\) over all regimes. The framework also considers threshold objectives such as \[ \Pr^\pi(Y>\lambda). \] This directly extends optimal treatment regime methodology from mean optimization to distributional criteria [1407.3414]. The underlying outcome model at stage 2 is \[ Y=m(\bH_2)+A_2 c(\bH_2)+\epsilon, \] with \(\epsilon\) independent of \((A_2,\bH_2)\). For threshold optimization, the optimal second-stage rule is \[ \pi_2^*(\bh_2)=\mathrm{sgn}\{c(\bh_2)\}, \] because \(\pi_2(\bh_2)c(\bh_2)\le |c(\bh_2)|\). The first-stage rule is more intricate: it depends on the full induced future distribution through \[ I\{y,F_\epsilon,G(\cdot,\cdot\mid \bh_1,a_1)\} = \int F_\epsilon(y-u-|v|)\,dG(u,v\mid \bh_1,a_1), \] where \(G(\cdot,\cdot\mid \bh_1,a_1)\) is the joint conditional distribution of \(m(\bH_2)\) and \(c(\bH_2)\) given first-stage history and treatment. The threshold-optimal first-stage rule is \[ \pi_{1,\lambda}^{\mathrm{TIQ}}(\bh_1) = \arg\min_{a_1} I\{\lambda,F_\epsilon,G(\cdot,\cdot\mid \bh_1,a_1)\}. \] This is the basis of TIQ-learning [1407.3414]. Quantile Interactive Q-learning (QIQ-learning) solves the harder quantile problem by searching over threshold-indexed TIQ rules. Defining \[ \Gamma(\bh_1,y)\triangleq \mathrm{sgn}\{d(\bh_1,y)\}, \] the method introduces \[ \tStar \triangleq \inf \left \{y: \Pr^{\Gamma(\cdot,y),\pi_2^*}(Y\le y)\ge \tau \right \}, \] and, under continuity and strict monotonicity of \(F_\epsilon\), uses \[ \pi_{1,\tau}^{\mathrm{QIQ}}(\bh_1)=\Gamma(\bh_1,\tStar). \] The resulting procedure estimates \(m\), \(c\), \(F_\epsilon\), and \(G\), computes \(\hat\tStar\), and recovers a regime whose attained quantile is consistent for the target. Theorem 2 establishes consistency of TIQ-learning for \(\Pr^\pi(Y>\lambda)\), and Theorem 3 establishes consistency of QIQ-learning for the target quantile [1407.3414]. This framework also clarifies why standard mean-based Q-learning is insufficient for quantile objectives. Quantiles are nonlinear functionals, so Bellman-style mean recursion does not directly apply. The paper further shows that replacing \(Y\) by \(\mathbbm 1_{Y>\lambda}\) in binary Q-learning does not generally recover a genuine threshold- or quantile-optimal rule; under its generative model, the resulting estimand reduces to the same object as mean-based Q-learning and can fail to vary with \(\lambda\) [1407.3414]. ## 4. Sequential classification, binary search, and discrete outcomes A recent single-stage development reformulates quantile OTR estimation as a sequence of weighted classification problems. The starting point is \[ d^*=\argmax_d Q_\tau\{Y(d)\}, \qquad Q_\tau\{Y(d)\}=\sup\{q:S(q,d)>1-\tau\}, \] with \[ S(q,d)=P\{Y(d)>q\}. \] For each fixed threshold \(q\), the inner problem is \[ d_q^*=\argmax_d S(q,d), \] and the optimal quantile value \(q^*\) is the largest threshold such that \[ \max_d S(q,d)\ge 1-\tau. \] The estimation strategy therefore uses an outer binary search over \(q\) and an inner classification problem for \(d_q^*\) [2507.11255]. The crucial representation is doubly robust: \[ S(q,d)=E\left[\psi(1,q;\pi^*,g^*)d(\boldsymbol{X})+\psi(0,q;\pi^*,g^*)\{1-d(\boldsymbol{X})\}\right], \] where \[ \psi(a,q;\pi^*,g^*)= \frac{I(Y>q)-g^*(q;\boldsymbol{X},a)}{\pi^*(a\mid \boldsymbol{X})}I(A=a)+g^*(q;\boldsymbol{X},a), \] and \[ g^*(q;\boldsymbol{X},a)=P(Y>q\mid \boldsymbol{X},A=a). \] This yields the weighted classification equivalence \[ d_q^* = \argmax_d S(q,d) = \argmin_d E\left[|C(\boldsymbol{X},q;\pi^*,g^*)|\,I\{Z^*\neq d(\boldsymbol{X})\}\right]. \] Using a decision function \(f\) with \(d(\boldsymbol X)=I\{f(\boldsymbol X)>0\}\) and hinge loss \(\phi(x)=(1-x)^+\), the estimator becomes \[ \hat{f}_q= \argmin_{f\in\mathcal{F}} \frac{1}{n}\sum_{i=1}^n |C(\boldsymbol{X}_i,q;\hat\pi,\hat g)| \, \phi\{(2\widehat Z_i-1)f(\boldsymbol{X}_i)\} +\lambda_n\|f\|^2, \] with \(\mathcal F\) taken from an RKHS and either a linear kernel or the Gaussian kernel \[ k_{\sigma_n}(\boldsymbol x,\boldsymbol z)=\exp\left(-\sigma_n^2\|\boldsymbol x-\boldsymbol z\|^2\right). \] The estimated survival under the estimated rule is evaluated through the smoothed plug-in estimator \[ \widehat{S}_{h_n}(q,\hat f_q)=\frac{1}{n}\sum_{i=1}^n \left[ \psi_i(1,q;\hat\pi,\hat g)\Phi\{\hat f_q(\boldsymbol X_i)/h_n\} + \psi_i(0,q;\hat\pi,\hat g)\{1-\Phi(\hat f_q(\boldsymbol X_i)/h_n)\} \right]. \] The resulting algorithm tunes \(\lambda_n\) and, for the Gaussian kernel, \(\sigma_n\) by \(5\)-fold CV, and recommends \(h_n=0.2/\log n\), \(\kappa_n=6^{-1}n^{-1/2}\operatorname{sd}(Y)\), and \(\epsilon_n=0.5n^{-1/2}\) [2507.11255]. A major contribution concerns discrete outcomes. For discrete \(Y\), the survival function \(S(q,d)\) is piecewise constant, the quantile is stepwise, direct empirical quantile maximization can be inconsistent, and tied quantile-optimal regimes raise an “ineffectiveness” issue. The proposed remedy linearly interpolates the survival function between adjacent support points \(v_1<\cdots1-\tau\}. \] The key theoretical inclusion is \[ \tilde{\mathcal D}^*\subseteq \mathcal D^*, \] so any regime maximizing the smoothed quantile also maximizes the original quantile. Under Assumption 9, which requires the sign of \[ P(Y>q\mid \boldsymbol X=\boldsymbol x,A=1)-P(Y>q\mid \boldsymbol X=\boldsymbol x,A=0) \] to remain invariant across \(q\in[v_1,v_l)\), Corollary 1 further gives \[ E\{Y(\hat d)\}-\max_d E\{Y(d)\}=o_p(1). \] Theorem 3 establishes quantile-value consistency in the continuous case, and Theorem 4 proves that for discrete outcomes \[ \lim_{n\to\infty}P\!\left[Q_\tau\{Y(\hat d)\}=\max_d Q_\tau\{Y(d)\}\right]=1. \] Simulation results favor QIQ-learning in a correctly specified linear setting, but SCL-Gaussian performs best in nonlinear continuous and discrete settings, and the ACTG175 analysis reports the highest estimated quantile values for SCL-Gaussian at \(\tau=0.25,0.50,0.75\) [2507.11255]. ## 5. Identification, partial identification, and minimax regret One important branch of the literature argues that individualized quantile treatment choice should be based on the conditional quantile of the individual treatment effect, \[ Q_\tau(Y_1-Y_0\mid X), \] rather than on the difference of conditional quantiles \[ Q_\tau(Y_1\mid X)-Q_\tau(Y_0\mid X). \] The reason is that the person at quantile \(\tau\) under treatment need not be the same person at quantile \(\tau\) under control, so \[ Q_\tau(Y_1\mid X)-Q_\tau(Y_0\mid X)\neq Q_\tau(Y_1-Y_0\mid X) \] in general. This reframes quantile OTRs as policies that maximize \[ E[\delta(X)Q_\tau(Y_1-Y_0\mid X)], \] with the unconstrained first-best rule \[ \delta_\tau^\dagger(x)=1\{Q_\tau(Y_1-Y_0\mid X=x)\ge 0\}. \] At \(\tau=0.5\), the median criterion admits a majority-benefit interpretation; for continuous \(Y_d\), \[ Q_{0.5}(Y_1-Y_0\mid X)\ge 0 \iff P[Y_1\ge Y_0\mid X]\ge 1/2. \] This makes the median QoTE rule a majority-vote rule in the sense formalized in Theorem 1 [2311.15878]. The difficulty is identification. Even with \(Y_d\perp D\mid X\), the distribution of \(Y_1-Y_0\mid X\) depends on the unobserved joint distribution of \((Y_1,Y_0)\mid X\), so the QoTE is generally only partially identified. The baseline bounds are Makarov-type bounds: \[ Q_\tau^L(x)\le Q_\tau(Y_1-Y_0\mid X=x)\le Q_\tau^U(x), \] with \[ Q_\tau^L(x)= \begin{cases} \inf_{u\in[\tau,1]}\left[Q_u(Y_1\mid X=x)-Q_{u-\tau}(Y_0\mid X=x)\right], & \tau\neq 0,\\ Q_0(Y_1\mid X=x)-Q_1(Y_0\mid X=x), & \tau=0, \end{cases} \] and \[ Q_\tau^U(x)= \begin{cases} \sup_{u\in[0,\tau]}\left[Q_u(Y_1\mid X=x)-Q_{1+u-\tau}(Y_0\mid X=x)\right], & \tau\neq 1,\\ Q_1(Y_1\mid X=x)-Q_0(Y_0\mid X=x), & \tau=1. \end{cases} \] Stronger assumptions such as positive dependence, joint conditional independence, deconvolution, Roy-type models, rank invariance, and symmetry can tighten or point identify the target. Under rectangularity of the identified set, minimax regret simplifies to maximization of \[ E[\delta(X)\bar Q_\tau(X)], \] where \[ \bar Q_\tau(x)=Q_\tau^U(x)1\{Q_\tau^L(x)\ge 0\} +Q_\tau^L(x)1\{Q_\tau^U(x)\le 0\} +\big(Q_\tau^U(x)+Q_\tau^L(x)\big)1\{Q_\tau^L(x)<00, \] so a natural quantile-targeted treatment rule is \[ d_\tau(x)=\mathbf{1}\{\Delta(x,\tau)>0\}. \] This is a direct implication of the treatment-comparison framework rather than a formal OTR theorem [2503.18523]. The methodology debiases the linear functional directly. For each treatment group, it constructs a projection direction \(\widehat M_{\tau,k}\) by constrained optimization, with the novel variance-enhancement constraint \[ \left| x_{\text{new}}^\top \widehat\Sigma_k M_{\tau,k} - \|x_{\text{new}}\|_2^2 \right| \le \|x_{\text{new}}\|_2^2 \lambda_k, \] and then defines the debiased estimator \[ \widehat{x_{\text{new}}^\top \beta_{\tau,k}} = x_{\text{new}}^\top \widehat\beta_{\tau,k} + \frac{1}{n_k} \widehat M_{\tau,k}^\top \sum_{i=1}^{n_k} \widehat\eta_{k,i}(\tau)\, X_{k,i}\, \varphi_\tau(Y_{k,i}-X_{k,i}^\top \widehat\beta_{\tau,k}), \] where \(\varphi_\tau(u)=\tau-\mathbf 1\{u\le 0\}\). This yields the IQTE estimator \[ \widehat\Delta_{\tau,\text{new}} = \widehat{x_{\text{new}}^\top \beta_{\tau,1}} - \widehat{x_{\text{new}}^\top \beta_{\tau,2}}. \] The paper proves pointwise asymptotic normality, \[ \frac{\widehat\Delta_{\tau,\text{new}}-\Delta_{\tau,\text{new}}}{\sqrt{\widehat V_\tau}} \xrightarrow{d} N(0,1), \] constructs confidence intervals and one-sided tests, establishes weak convergence over \(\tau\), and derives minimax-optimal rates for expected CI length and testing detection boundary. This makes it a rigorous inferential module for quantile-sensitive treatment comparison in high-dimensional settings, even though it is not itself an OTR learning paper [2503.18523]. Across these strands, several recurring distinctions structure the subject. First, mean-optimal and quantile-optimal rules need not coincide: treatments can share the same conditional mean and differ in lower or upper quantiles, and simulations in both the robust-regression and dynamic-learning literatures show that rules tuned to \(\tau=0.25\), \(\tau=0.5\), or other targets can prioritize materially different parts of the outcome distribution [1604.03648] [1407.3414]. Second, quantile optimization is not merely “robust mean regression”: pinball-loss OTRs can be exact or approximate quantile procedures depending on the model and independence assumptions, and dynamic quantile regimes require distributional recursion rather than ordinary mean-based Bellman updates [1604.03648] [1407.3414]. Third, direct quantile value search can be computationally unstable or inconsistent for discrete outcomes, which motivates classification-based reformulations and smoothing constructions [2507.11255]. Fourth, quantile criteria can be fundamentally harder than mean criteria under worst-case finite-sample decision theory, because quantiles are discontinuous functionals of the distribution and can make minimax regret largely uninformative [2601.03428]. Taken together, the literature presents quantile OTRs as a family of regime-learning problems in which the target is a chosen feature of the outcome distribution rather than the expectation. The main lines of development are robust single-stage contrast modeling through pinball loss, dynamic distributional learning through TIQ- and QIQ-learning, sequential classification with binary search and doubly robust survival representations, partial-identification and minimax-regret policies based on quantiles of individual gains, and high-dimensional inference for individualized conditional quantile contrasts. This suggests that the defining issue in quantile OTRs is not only which algorithm is used, but also which quantile object is being optimized and under what causal, structural, and decision-theoretic assumptions [1604.03648] [1407.3414] [2507.11255] [2311.15878] [2503.18523] [2601.03428]. Source: https://www.emergentmind.com/topics/quantile-optimal-treatment-regimes-otrs