--- title: 'Optimal Treatment Regime: Foundations & Methods' url: https://www.emergentmind.com/topics/optimal-treatment-regime-otr type: topic --- # Optimal Treatment Regime: Foundations & Methods An optimal treatment regime (OTR)—also called an optimal treatment rule, individualized treatment rule, or policy—is a decision rule that assigns treatment on the basis of covariates so as to maximize a population-level criterion, typically an expected outcome. In the standard static binary-treatment setup, with covariates \(X\), treatment \(A\in\{0,1\}\), outcome \(Y\), and potential outcomes \(Y(1),Y(0)\), a regime \(d:\mathcal X\to\{0,1\}\) induces \(Y(d)=Y(1)d(X)+Y(0)\{1-d(X)\}\) and value \(V(d)=E[Y(d)]\); the OTR is any \(d^{\mathrm{opt}}\in\arg\max_d V(d)\). Contemporary work extends this formulation to dynamic regimes, alternative value functionals such as conditional medians or survival probabilities, observational settings with nuisance estimation, and nonregular inference problems in which the optimal rule itself may be difficult to estimate or summarize [2405.16161; 2401.10867]. ## 1. Formal definition and causal structure In the static potential-outcomes formulation, the observed data are i.i.d. copies of \((X,A,Y)\), where \(X\in\mathcal X\subset\mathbb R^l\), \(A\in\{0,1\}\), and \(Y\in\mathbb R\). A regime is a mapping \[ d:\mathcal X\to\{0,1\}, \] and its value is \[ V(d)=E[Y(d)]=E\big[Y(1)d(X)+Y(0)\{1-d(X)\}\big]. \] Under unconfoundedness or ignorability, \[ \{Y(0),Y(1)\}\perp A\mid X, \] consistency, \[ Y=Y(1)A+Y(0)(1-A), \] and positivity, \[ 00\}\) | [2405.16161] | | Conditional-median regime | \(\psi_d=E_X[m(Y^d\mid X)]\) | [2103.01802] | | \(t\)-year survival regime | \(S^*(t;\eta)=P\{T^*(g_\eta(X))>t\}\) | [1407.7820] | | Treatment-initiation regime | \(V(d)=E[m\{d(X),X\}]\) with RMRL at \(a_0\) | [2107.04839] | | Dynamic welfare | \(W_{\boldsymbol\delta}=E[Y_T(\boldsymbol\delta(\cdot))]\) | [1912.10014] | | Indefinite-horizon survival regime | \(\mathcal V(\pi)=E^\pi(T\wedge\tau)\) | [2501.18070] | A major structured subclass is the linear regime \[ d(X;\beta)=\mathbb I\{X^\top\beta>0\}, \] usually indexed over a compact or normalized parameter space such as \[ B=\{\beta\in\mathbb R^l:\|\beta\|=1\}. \] This class is attractive because signs and magnitudes of \(\beta_j\) directly encode which covariates push treatment toward \(A=1\) or \(A=0\), and because the rule can be communicated as a weighted score. The resulting trade-off is explicit: nonlinear classes can approximate the truly optimal rule more closely, but linear rules improve interpretability, bedside usability, and inferential tractability [2405.16161]. Alternative criteria materially change the regime. In the median-based formulation, the rule is \[ d^*_{\text{ACME}}(X)=\mathbf 1\{m_1(X)>m_0(X)\}, \] where \(m_a(X)\) is the conditional median under treatment \(a\), and the criterion is the Average Conditional Median Effect \[ \psi_d=E_X[d(X)m_1(X)+(1-d(X))m_0(X)]. \] This criterion is within-group and median-based, rather than mean-based or based on the marginal median of \(Y^d\) [2103.01802]. Survival settings also diversify the objective. One strand maximizes \(t\)-year survival probability \(S^*(t;\eta)\) over a linear class of rules [1407.7820]. Another treats treatment timing itself as the decision and evaluates regimes through restricted mean residual lifetime at a prespecified \(a_0\), \[ m(a,x)=\int_{a_0}^{\tau} P\{T^*(a)\ge t\mid T^*(a)\ge a_0,X=x\}\,dt, \] with value \(V(d)=E[m\{d(X),X\}]\) [2107.04839]. In dynamic IV-based settings, optimality may become set-valued: when welfare is only partially identified, the identified set of optimal regimes consists of the maximal elements in a sharp partial welfare order rather than a unique rule [1912.10014]. These formulations imply that OTR is not a single estimator or algorithm but a family of decision problems. What changes across papers is the welfare functional being optimized, the structural assumptions under which it is identified, and the extent to which interpretability is imposed ex ante through the regime class. ## 3. Estimation paradigms The broader OTR literature includes regression-based methods such as Q-learning and A-learning, direct policy-search methods based on IPW or AIPW value estimation, TMLE-based policy learning, outcome-weighted learning, and modern RL/OPE/OFL approaches [2405.16161]. Across these paradigms, the core estimation problem is to recover either the rule itself, the value of a candidate rule, or both. For linear static regimes in observational data, a central direct-value strategy uses the augmented inverse probability weighted representation. With propensity score \(e(X)=P(A=1\mid X)\), outcome regressions \(\mu_a(X)=E[Y\mid X,A=a]\), and \[ \rho(A\mid X)=e(X)A+\{1-e(X)\}(1-A), \] the value of \(d(X;\beta)\) can be written as \[ V(\beta)=E\big[v(X,A,Y;\beta)\big], \] where \[ v(X,A,Y;\beta)=\frac{\mathbb I\{A=d(X;\beta)\}}{\rho(A\mid X)}\{Y-\mu_d(X;\beta)\}+\mu_d(X;\beta). \] Replacing \(e,\mu_a\) by estimators yields \[ \hat V_n(\beta)=\frac{1}{n}\sum_{i=1}^n \hat v(X_i,A_i,Y_i;\beta), \] and the optimal linear regime is estimated by \[ \hat\beta=\arg\max_{\beta\in B}\hat V_n(\beta),\qquad \hat d(X)=\mathbb I\{X^\top\hat\beta>0\}. \] In the cited implementation, the optimization over the unit sphere is carried out by a genetic algorithm via `rgenoud` [2405.16161]. Median-optimal regimes replace conditional means by conditional medians and estimate the conditional median treatment effect \[ \gamma(X)=m_1(X)-m_0(X). \] The learned rule is \[ \hat d^*(X)=\mathbf 1\{\hat\gamma(X)>0\}. \] The associated ACME estimator is influence-function based and doubly robust-style: \[ \hat\psi_{d,\mathrm{dr}}=\mathbb P_n\left[ \frac{\mathbf 1\{A=d(X)\}}{A\hat\pi_1(X)+(1-A)\hat\pi_0(X)} \frac{\tfrac12-\mathbf 1\{Y\le \hat m_A(X)\}}{\hat f_{A,\hat m}(X)}+\hat m_d(X)\right], \] while \(\hat\gamma_{\mathrm{dr}}(x)\) is obtained by regressing a pseudo-outcome \(\hat g(Z)\) on \(X\) through a linear smoother [2103.01802]. When unmeasured confounding precludes standard ignorability, proximal causal learning replaces \(Y(a)\perp A\mid X\) by proxy-based identification. With \(L=(X,W,Z)\), where \(W\) and \(Z\) are outcome-inducing and treatment-inducing confounding proxies, the value of a \(Z\)-based regime can be identified through an outcome confounding bridge \(h(w,a,x)\), \[ V(d_z)=E[h(W,d_z(X,Z),X)], \] and the value of a \(W\)-based regime through a treatment confounding bridge \(q(z,a,x)\), \[ V(d_w)=E\big[Y\,q(Z,A,X)\,\mathbf 1\{d_w(X,W)=A\}\big]. \] The cited paper further defines an expanded class \[ d_{zw}^{\pi}(X,W,Z)=\pi(X)d_z(X,Z)+(1-\pi(X))d_w(X,W), \] and estimates bridge functions with neural moment methods, followed by weighted classification steps for \(d_z\), \(d_w\), and a learned predilection function \(\pi(X)\) [2212.09494]. Longitudinal EHR settings create an additional nuisance process: visit times. In that case, single-stage individualized treatment rules can be estimated via doubly weighted dWOLS using both an inverse probability of treatment weight and an inverse intensity of visits weight, \[ W_i(t)=w_i(t;\hat\kappa)\rho_i(t;\hat\gamma), \] within weighted estimating equations for a blip model \[ b\big(\mathbf X_i^\psi(t)\big)=\psi^\top \mathbf X_i^\psi(t). \] The resulting rule is \[ \hat d(\mathbf Q_i(t))=\mathbb I\{\hat\psi^\top\mathbf X_i^\psi(t)\ge 0\}, \] and the method is designed to break the collider-stratification bias induced by covariate-driven observation times [2202.09611]. For genuinely dynamic regimes, one prominent modern strategy is backward recursion with doubly robust pseudo-outcomes. At a single time point, the DR transformation \[ D(Q,g)(O)=\frac{2A-1}{g(A,H)}\big(Y-Q(H,A)\big)+Q(H,1)-Q(H,0) \] satisfies \[ E[D(Q,g)(O)\mid V]=B(V), \] where \(B(V)=E[Y_1-Y_0\mid V]\) is the blip. Repeating this backwards over time yields a sequence of estimated rules \[ d_{t,\mathrm{opt}}(V_t)=\mathbb I\{\tilde B_t(V_t)>0\}, \] thereby learning an ODTR by dynamic programming with DR pseudo-outcomes rather than via a fully parametric Q-function [2401.10867]. Across these approaches, a common architecture emerges: identify a policy value; orthogonalize or augment it against nuisance estimation; restrict or regularize the policy class; and then optimize the estimated value. The divergence across papers lies mainly in how identification is achieved and how much structure is imposed on the decision rule. ## 4. Inference, nonregularity, and asymptotic phenomena Inference for OTRs is not a routine by-product of policy estimation. The central obstacle is that the rule itself is often defined by a non-smooth sign or indicator operation, and the difficulty is more acute when the optimal regime is nonunique. For optimal linear regimes estimated by AIPW maximization, the value estimator remains regular: \[ \sqrt n\{\hat V_n(\hat\beta)-V(\beta_0)\}\overset d\to \mathcal N(0,\sigma^2), \] but the regime parameter does not. Under additional smoothness, density, and curvature assumptions, \[ \|\hat\beta-\beta_0\|=O_p(n^{-1/3}), \] and \[ n^{1/3}(\hat\beta-\beta_0)\overset d\to \arg\max_{t\in\mathbb R^l}\Big\{-\tfrac12 t^\top H t + W(t)\Big\}, \] that is, the argmax of a mean-zero Gaussian process with a quadratic drift. The nonregularity comes from the indicator \(\mathbb I\{X^\top\beta>0\}\), which creates a kink in the objective. Standard nonparametric bootstrap is therefore invalid for \(\hat\beta\), and the paper adapts the Cattaneo–Jansson–Nagasawa bootstrap by centering and drift-adjusting the objective with an estimated Hessian \(H_n\) [2405.16161]. A distinct but related nonregularity appears in inference for the mean outcome under the OTR itself. When \[ P\{\tau(X)=0\}>0, \] the regularity assumption that the OTR is unique fails, classical inference that treats the estimated OTR as if it were the true OTR can be biased, and pathwise differentiability of the target can fail except in a degenerate deterministic-outcome case [2509.09773]. The proposed remedy is adaptive smoothing of the estimated rule: \[ d_s\big(X;\hat\tau,h_n,t_0\big), \] with \(t_0\) chosen adaptively as the propensity score \(\pi(1,X)\). The resulting estimator is valid regardless of whether regularity holds, achieves asymptotic normality, and attains a derived lower bound on the asymptotic variance for the class of robust asymptotically linear unbiased estimators. In that sense, the procedure is efficient not only in the classical regular case but also in the explicitly nonregular setting [2509.09773]. Survival-timing OTRs generate yet another asymptotic pattern. When the decision variable is a continuous initiation time and the value is a kernel-based estimator of restricted mean residual lifetime, the regime parameter is asymptotically normal at rate \[ (n|\mathbf h_1|h_2^3)^{1/2}, \] whereas the optimized value has a nonstandard limit: \[ n|\mathbf h_1|h_2^3\{M_n(\hat\beta^{\mathrm{opt}})-M(\beta^{\mathrm{opt}})\}\xrightarrow d \sum_{i=1}^s \frac{a_i}{2}\chi^2(r_i). \] The weighted chi-squared limit arises because continuous treatment timing requires kernel smoothing in both \(X\) and \(A\), and the optimized value is dominated by quadratic fluctuation of the estimated parameter [2107.04839]. A recurrent misconception is that value estimation is always easier than regime estimation. The recent literature makes a sharper distinction: sometimes the value remains regular while the rule is nonregular; sometimes the rule can be estimated consistently but the optimized value has a nonstandard law; and when uniqueness of the OTR fails, even inference on the value can require specialized smoothing or resampling. This separation between inference on the policy and inference on the value is now a defining technical feature of OTR methodology. ## 5. Extensions beyond the standard static binary-treatment model A substantial part of the modern OTR literature is devoted to departures from the canonical single-population, single-stage, no-unmeasured-confounding setting. When sequential ignorability of treatment fails in dynamic settings, IV-based methods replace point identification of welfare by partial identification. Under a sequential IV assumption, \[ Z_t \perp \big(\boldsymbol Y(\boldsymbol d),\boldsymbol D(\boldsymbol z)\big)\mid \boldsymbol Z^{t-1}, \] welfare differences between dynamic regimes can be bounded by linear programs over a latent-state distribution \(q\). The identified set of optimal regimes is then \[ \mathcal D_p^*=\{\boldsymbol\delta_{k'}(\cdot):L_{k,k'}\le 0,\ \forall k\neq k'\}, \] the set of maximal elements of a sharp partial welfare order, rather than a single regime [1912.10014]. When observation times are covariate-driven, as in EHR data, standard DTR assumptions fail because conditioning on visit occurrence opens collider paths. The proposed remedy is a repeated-measures ITR estimated by dWOLS with both IPT and inverse-intensity-of-visits weighting, under assumptions such as \[ E[dN_i(t)\mid V_i(t)] = \xi_i(t)\exp\{\gamma^\top V_i(t)\}d\Lambda_0(t). \] This explicitly adjusts the observation process jointly with the treatment process [2202.09611]. When measured covariates are insufficient to remove confounding, proximal causal learning uses confounding proxies \(W\) and \(Z\) together with bridge functions \(h\) and \(q\) to identify regime values and construct optimal regimes in classes indexed by \((X,Z)\), \((X,W)\), or by adaptive combinations of both: \[ d_{zw}^{\pi}(X,W,Z)=\pi(X)d_z(X,Z)+(1-\pi(X))d_w(X,W). \] The resulting proximal regime dominates earlier proximal policies that globally selected either the \(Z\)-based or \(W\)-based policy class [2212.09494]. Transportability problems arise when the source and target populations differ in covariate distribution and only target summary statistics are available. In that setting, calibrated AIPW estimators replace the usual source-sample average by weighted averages over source units, with calibration constraints \[ \sum_{i=1}^n w_i\{g(X_i)-\mu_{g0}\}=0,\qquad \sum_{i=1}^n w_i=1. \] The learned regime \[ \hat\beta^{\mathrm c}=\arg\max_\beta \hat V^{\mathrm c}(\beta) \] targets a pseudo-population that can coincide with the target population when the calibration weights recover the density ratio \(f^{\mathrm t}(X)/f^{\mathrm s}(X)\) [2201.06229]. Survival and longitudinal RL formulations further broaden the field. Indefinite-horizon DTRs for censored survival outcomes can be estimated by generalized survival random forests that repeatedly apply a single rule \[ \pi:\mathcal H\to\mathcal A \] over pooled visits, while optimizing truncated mean survival \[ \mathcal V(\pi)=E^\pi(T\wedge\tau). \] The method uses summarized histories of fixed dimension, backward survival-curve augmentation, and iterative refitting, and it allows patients to have different numbers of decision points [2501.18070]. Continuous-time timing decisions produce another generalization: treatment initiation itself becomes the action \(d(X)\in[0,a_0]\), and the regime is chosen to maximize expected restricted mean residual lifetime rather than a binary treatment contrast [2107.04839]. These extensions show that the notion of an OTR survives substantial relaxation of the standard setup, but the meaning of “optimal” often changes with identification strength. Under ignorability it is typically a maximizer of an identified value; under IV assumptions it may be an identified set; under transportability it may be optimal in a calibrated pseudo-population; and under nonregularity its value may require specialized inference even when the rule class is simple. ## 6. Interpretation, applications, and methodological tensions A major motivation for restricting OTRs to interpretable classes is substantive interpretability. In the linear-rule setting, \[ \hat d(X)=\mathbb I\{\hat\beta_0+\hat\beta_1 X_1+\cdots+\hat\beta_l X_l>0\}, \] the sign of \(\hat\beta_j\) indicates whether larger \(X_j\) pushes toward treatment \(A=1\) or control \(A=0\), and the magnitude \(|\hat\beta_j|\) reflects relative importance for the decision boundary. This is why the 2024 linear-regime paper places particular emphasis on inference for the regime parameter itself rather than only on the value [2405.16161]. In the eICU vasopressor application, that interpretive emphasis yields clinically legible findings. With \(\epsilon_n=0.3\), admission temperature had a positive coefficient with 95% CI excluding zero, reported as Temp: estimate \(0.424\), CI \((0.004,0.637)\); with \(\epsilon_n=0.5\), WBC had a positive coefficient with 95% CI excluding zero, reported as WBC: estimate \(0.486\), CI \((0.038,0.971)\). The interpretation offered is that fever and leukocytosis tilt the optimal decision rule toward vasopressor use when optimizing fluid balance [2405.16161]. Other applications illustrate how different value criteria change substantive recommendations. In the ACTG 175 HIV trial, the median-optimal policy had the highest estimated ACME, slightly outperforming treat-all and mean-optimal policies, while the motivating female subgroup histograms showed a large mean difference but almost no median difference between treated and control outcomes, exactly the situation in which median-based OTRs are intended to differ from mean-based rules [2103.01802]. In the oropharynx cancer study, a Bayesian loss on the bivariate potential outcomes was used to avoid unnecessary chemotherapy burden. Under the OTR.25 loss, the proposed regime reduced the frequency of CRT assignment by approximately 75% without reducing the average survival probability, thereby changing the rule not by improving survival per se but by reweighting survival against treatment burden [1809.06679]. Dynamic applications make the same point in sequential settings. In the buprenorphine–naloxone analysis, the learned ODTR outperformed a clinically defined dose-escalation strategy when evaluated by the week-6 risk of return-to-regular-opioid-use, illustrating how DR backward recursion can yield a clinically interpretable dynamic rule that differs from standard practice [2401.10867]. In the antidepressant BMI study using CPRD, the doubly weighted dWOLS estimator identified effect modification not detected by simpler estimators, and applying the estimated rule yielded about a \(0.7\)–\(0.8\) point increase in the BMI-related utility on a roughly 100-point scale [2202.09611]. In pediatric Crohn’s disease, the indefinite-horizon survival-forest regime improved restricted mean hospital-free survival relative to observed practice, with the two-strata implementation increasing mean survival by about 30 days [2501.18070]. The principal methodological tension throughout the literature is between flexibility and interpretability. Flexible rules—trees, RKHS-based learners, deep nets, or pooled forest-based DTRs—can achieve better approximation of the optimal value, but often at the cost of harder inference, greater sensitivity to nuisance estimation, and weaker clinical transparency. Structured rules—linear scores, decision lists, or single-index blips—sacrifice approximation flexibility in exchange for direct substantive interpretation and, in some settings, tractable asymptotics. A second tension concerns identification: stronger assumptions such as sequential ignorability yield point-identified optimal policies, whereas weaker assumptions such as IV or proximal identification can produce only partial orders, pseudo-population optimality, or sensitivity-dependent recommendations. The contemporary OTR literature can therefore be read as a sequence of answers to the same question—what treatment rule is best?—posed under progressively different assumptions about outcomes, observability, transportability, and inferential regularity. Source: https://www.emergentmind.com/topics/optimal-treatment-regime-otr