Policy-DRIFT: Drift-Centered Policy Constructions

• Policy-DRIFT is a family of drift-centered policy constructions that explicitly models drift as a primary variable across diverse settings.
• It decouples reward-guided target selection from policy gradient tracking, leading to improved performance and energy efficiency in systems like turbulent flow control and RL.
• The approach spans applications from robust offline policy learning under concept drift to on-policy distillation in language models, emphasizing physical plausibility and data freshness.

Searching arXiv for exact and adjacent uses of “Policy-DRIFT” to ground the article in relevant papers. Policy-DRIFT is a non-standard research label applied to several distinct but structurally related ideas in machine learning, control, and information systems. In its most literal usage, it denotes a generative control framework for turbulent drag reduction that relocates reward information from policy gradients to generative-model inference (Mahajan et al., 13 May 2026). In adjacent literatures, the same label or closely related formulations denote distributionally robust policy learning under concept drift (Wang et al., 2024), drift-based one-step policy learning in offline and online reinforcement learning (Houssaini et al., 29 May 2026), blockwise policy-drift gating for on-policy distillation (Zheng et al., 23 Jun 2026), and meta-learned drift functions inside Mirror Learning (Lu et al., 2022). This suggests that “Policy-DRIFT” is best understood as a family of drift-centered policy constructions rather than a single standardized algorithm.

1. Scope and nomenclature

Across the literature, “drift” refers to different objects: temporal displacement in archive browsing, concept drift in outcome mechanisms, transport fields over action distributions, deviations between behavior and current policies, or reliability-regime changes in human-in-the-loop systems. The common thread is that policy behavior is shaped by an explicitly modeled drift process rather than by unconstrained optimization alone (Mahajan et al., 13 May 2026).

Research context	Representative paper	Meaning of drift
Turbulent flow control	"Policy-DRIFT: Dynamic Reward-Informed Flow Trajectory Steering" (Mahajan et al., 13 May 2026)	Reward-guided steering on a generative flow manifold
Concept-drift policy learning	"Distributionally Robust Policy Learning under Concept Drifts" (Wang et al., 2024)	Conditional reward-law perturbation with fixed covariate law
One-step generative RL	"Drift Q-Learning" (Houssaini et al., 29 May 2026)	Action-space drift field for behavioral regularization
On-policy distillation	"Blockwise Policy-Drift Gating for On-Policy Distillation" (Zheng et al., 23 Jun 2026)	Old/current student log-probability shift on reused rollouts
Mirror Learning	"Discovered Policy Optimisation" (Lu et al., 2022)	Learned drift function shaping policy updates
Web archiving	"Evaluating Sliding and Sticky Target Policies..." (Ainsworth et al., 2013)	Temporal drift induced by archive target-datetime policies

The term therefore spans at least three major technical regimes. In one regime, drift is an operator over probability distributions or action samples. In a second, it is a distribution shift or reliability shift in the environment. In a third, it is a deviation signal used to regulate training on stale data. The resulting methods are not interchangeable, but they share a design pattern: drift is elevated from a nuisance variable to the primary policy object.

2. Dynamic Reward-Informed Flow Trajectory Steering

The most direct use of the title “Policy-DRIFT” is the wall-bounded turbulence framework introduced for active drag reduction in turbulent channel flow at friction Reynolds number $\mathrm{Re}_\tau = 180$ (Mahajan et al., 13 May 2026). Its central claim is architectural: reward should not directly shape policy gradients. Instead, a conditional flow matching model constructs a physically grounded manifold of realizable future flow states, Terminal Reward Guidance steers inference on that manifold toward reward-maximizing targets, and a lightweight DRL policy is trained only to track those targets via root-mean-squared error minimization.

The conditional flow model uses the path

$$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$

with

$$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$

and learns a velocity field by minimizing

$$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$

Reward guidance is based on the cost-aware terminal objective

$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$

where

$$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$

The guidance step is applied in pre-placement form,

$$\tilde{\mathbf{u}}_s^{+} = \tilde{\mathbf{u}}_s + \gamma\,\delta s\;\nabla_{\tilde{\mathbf{u}}_s} R_\psi(\tilde{\mathbf{u}}_s,\, s),$$

followed by transport through the learned flow model. The pre-placement design is explicitly motivated by physical realizability: the nudged state is passed back through the learned manifold projector, rather than accepted as a free-form reward-hacked terminal state.

The implementation uses a 3D U-Net with about 23.4M parameters operating on inputs of shape $[B,7,16,16,64]$. Training data comprise consecutive snapshot pairs from uncontrolled flow, opposition control, and DRL wall-shear-stress control, with 7,350 pairs per subset and 22,050 total. At deployment, the controller observes only wall-parallel sensing variables $u'^+$ and $v'^+$ at $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$0, generates a target one horizon ahead, and tracks that target over 8 actuation steps per horizon (Mahajan et al., 13 May 2026).

The reported quantitative outcome is 48.95% drag reduction, with $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$1 and $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$2. The same table reports 42.13 drag reduction for TD3-WSE and 35.23 for opposition control, and the abstract states that Policy-DRIFT is about 16.2% higher than the DRL benchmark and uses about 37× less actuation energy (Mahajan et al., 13 May 2026). The significance of this result is not merely numerical. The framework reassigns reward from policy-gradient estimation to guided generative inference, thereby decoupling target selection from the tracking policy and treating physical plausibility as a hard inductive constraint.

3. Policy learning under concept drift

A different use of Policy-DRIFT appears in distributionally robust offline policy learning when only the conditional reward mechanism is allowed to drift (Wang et al., 2024). Here the defining modeling choice is to leave the marginal covariate law $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$3 untouched and to robustify only the conditional outcome distribution. The data are i.i.d. samples $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$4 under unconfoundedness, overlap, and bounded rewards, and the robust value of a target policy $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$5 is

$$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$6

with the uncertainty set defined through a KL ball over conditional outcome distributions. This explicitly avoids the joint-distribution framework used by earlier robust policy learning methods.

The analysis uses strong duality to rewrite the inner infimum in terms of nuisance functions $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$7, then builds a cross-fitted doubly robust estimator. The estimator remains asymptotically normal even when nuisance estimators converge slower than $$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$8, provided the product-rate condition

$$p_s(\tilde{\mathbf{u}} \mid \mathbf{u}_1) = \mathcal{N}\!\left(\tilde{\mathbf{u}};\; s\,\mathbf{u}_1,\; (1-s)^2 I\right),$$9

holds and the dual nuisance estimator is faster than $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$0 in $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$1. Policy learning proceeds by maximizing the estimated robust value over a class $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$2, with finite-sample regret of order

$$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$3

where $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$4 is the Hamming entropy integral, and the paper gives a matching lower bound up to logarithmic factors (Wang et al., 2024).

The numerical study uses a multi-action contextual bandit with $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$5, $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$6, random forests for propensity and regression estimation, cubic splines plus Nelder–Mead for the dual ERM step, and policytree for final policy optimization. The benchmark is SNLN from Si et al. (2023), adapted to the concept-drift setting, and the reported empirical outcome is that Policy-DRIFT achieves substantially higher robust policy values and higher worst-case rewards under KL-sphere testing designed to mimic concept drift (Wang et al., 2024).

A related but distinct policy problem under concept drift is adaptive model retraining under a hard average resource budget. RCCDA formulates update timing as a causal threshold policy using current loss, best historical loss, an estimated gradient norm, and a virtual queue, with the threshold

$$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$7

and proves asymptotic compliance with the update-cost budget (Piaseczny et al., 30 May 2025). This suggests a broader drift-policy theme: once the source of drift is specified, robustness can be localized to the mechanism that actually changes.

4. Drift as action-space transport in reinforcement learning and robotics

Another major lineage uses drift not as environmental non-stationarity but as a training-time transport operator over policy outputs. Drift Q-Learning, or Policy-DRIFT in the paper’s terminology, addresses offline reinforcement learning from a fixed dataset by combining a drift-based behavioral regularizer with critic-driven policy improvement (Houssaini et al., 29 May 2026). The actor remains stochastic,

$$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$8

and for each state generates $$\tilde{\mathbf{u}}_s = (1-s)\boldsymbol{\eta} + s\,\mathbf{u}_1,\qquad \boldsymbol{\eta}\sim\mathcal{N}(\mathbf{0},I),$$9 candidate actions. The drift field is

$$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$0

with attraction

$$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$1

toward the single dataset action and repulsion

$$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$2

based on softmax-normalized Gaussian-kernel logits. The actor minimizes

$$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$3

where the drift term keeps the policy on support and the $$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$4-term biases it toward higher-value regions. The method is a single-network, single-pass generator; the abstract reports strong performance on D4RL and OGBench, and the details report inference roughly $$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$5 faster than FQL, $$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$6 faster than Diffusion-QL, and $$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$7 faster than IDQL and IFQL (Houssaini et al., 29 May 2026).

In online robot control, Drift-Based Policy Optimization uses a two-stage construction: Drift-Based Policy (DBP) first internalizes refinement through a fixed-point drifting objective, and DBPO then adds a stochastic interface that makes on-policy PPO updates exact with respect to a stored latent variable $$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$8 (Gao et al., 4 Apr 2026). The one-step generator is

$$\mathcal{L}_{\mathrm{CFM}}(\theta)= \mathbb{E}\!\left[ \left\|v_\theta(\tilde{\mathbf{u}}_s,\, s,\, \mathbf{u}_0) - (\mathbf{u}_1 - \boldsymbol{\eta})\right\|^2 \right].$$9

and training regresses outputs toward a stop-gradient drift-corrected target. The online interface defines

$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$0

so that PPO ratios are exact when the same latent $$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$1 is reused from the rollout buffer. The paper reports that DBP improves average success from 0.79 to 0.83 on a reproduced diffusion-policy suite while reducing inference from 100 NFE to 1 NFE, and on a real dual-arm UR5 setup achieves 105.2 Hz control with average end-to-end latency of about 9.5 ms (Gao et al., 4 Apr 2026).

Drifting Field Policy places the update directly in probability space by treating policy improvement as a reverse-KL Wasserstein-2 gradient flow toward a soft target policy (Koo et al., 8 May 2026). The target policy

$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$2

induces a drift field whose small-bandwidth limit decomposes into a $$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$3-ascent term and an anchor-policy score-matching term. Because the exact target is intractable, the method uses a top-$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$4 critic-selected surrogate and an actor loss

$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$5

The reported result is 95.8% average success, best on 9 of 12 tasks, with large gains over MVP on difficult cube-manipulation tasks (Koo et al., 8 May 2026).

Drift Flow Matching generalizes one-step drift models into a two-time transport framework that can also be iterated at test time (Ma et al., 17 May 2026). Instead of regressing only a terminal map, it learns

$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$6

between arbitrary times $$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$7, and trains with the stop-gradient loss

$$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$8

At one step it behaves like a drift model; with more steps it gains flow-matching-style test-time scaling. On ImageNet $$R(\hat{\mathbf{u}}_1)=\mathrm{DR}(\hat{\mathbf{u}}_1)-E_{\mathrm{act}}(\hat{\mathbf{u}}_1),$$9, DFM L/2 improves from 1.52 FID at NFE 1 to 1.31 at NFE 10, and on ToolHang state success increases from 0.41 at NFE 1 to 0.86 at NFE 10 (Ma et al., 17 May 2026).

Taken together, these methods define a coherent subfield in which drift is an explicit transport field over policy outputs. The unifying move is to shift iterative refinement from deployment-time denoising or ODE integration into the training objective, while preserving multimodality and support control.

5. Policy-drift as a freshness signal in language-model training

In on-policy distillation for reasoning models, Policy-DRIFT is a lightweight gating mechanism for reused rollouts (Zheng et al., 23 Jun 2026). The setting involves a behavior student that generated the response trajectory and a current student that is being optimized over multiple epochs on the same data. The method computes old/current log-probability shifts on the sampled token path, aggregates them over local blocks or spans, converts the aggregate into a detached gate

$$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$0

mean-normalizes the gate over valid response tokens,

$$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$1

and uses it to reweight the position-wise OPD loss: $$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$2 The paper studies fixed 64-token blocks and newline-delimited spans, with Block64 as the main default.

The experimental setting uses a Qwen3-1.7B-Base student, a Qwen3-4B-Base-GRPO teacher, a uniform 200-step training budget, and pass@8 as the primary solve-rate metric across AIME24, AIME25, MATH500, and AMC23. The headline result is that fixed 64-token block gating improves sampled-token OPD mean pass@8 from 0.4978 to 0.5160, and on Teacher-TopK/LSM the best trained student is LSM + Block64 with mean pass@8 of 53.3 (Zheng et al., 23 Jun 2026). The method is explicitly orthogonal to teacher-support matching: it does not change teacher targets, teacher Top-$$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$3 supports, teacher normalization, or rollout policy.

Closely related work on diffusion-policy RL post-training diagnoses a “double-drift” phenomenon: the ELBO can drift away from the true log-likelihood, and the resulting proxy gradient can then drift away from the true policy gradient of expected return (Jiang et al., 11 Jun 2026). DiPOD addresses this by interleaving self-distillation with policy-improving updates, or practically by adding an on-policy ELBO regularizer to each update. The reported gains are especially large on Countdown and Sudoku, where SPG + DiPOD reaches 80.08 and 97.56, versus 51.95 and 25.12 without DiPOD (Jiang et al., 11 Jun 2026).

A nearby LLM-RL formulation, Extreme Region Policy Distillation, separates aggressive off-policy signal extraction from KL-efficient trust-region distillation (Chen et al., 25 May 2026). Its central empirical observation is that extensive off-policy optimization spends much of its KL budget on unnecessary drift rather than genuine improvement. This supports the same general interpretation underlying Policy-DRIFT gating: local policy drift can be a useful control signal, but only if it is treated as a signal about data freshness or optimization geometry, not as a reward in itself.

6. Antecedents and related meanings

Important antecedents predate the current usage of the label. In web archiving, temporal drift was studied through two target-datetime policies: Sliding Target, used by archive interfaces such as the Wayback Machine, and Sticky Target, represented by the Memento API (Ainsworth et al., 2013). The study attempted 200,000 acyclic walks, following up to 50 links each, and found that Sticky Target held drift to less than 30 days on average regardless of walk length or number of domains visited, while generally producing at least 30 days less drift than Sliding Target. Here “policy drift” is not a learning update but a semantic consequence of UI design.

In reinforcement learning theory, drift entered policy optimization explicitly through Mirror Learning. "Discovered Policy Optimisation" meta-learns a drift function $$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$4 subject to non-negativity and zero-gradient conditions at the identity policy, producing Learnt Policy Optimisation (LPO) and then the closed-form Discovered Policy Optimisation (DPO) (Lu et al., 2022). DPO uses the piecewise drift

$$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$5

with $$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$6 and $$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$7. The resulting update rule is asymmetric relative to PPO and is interpreted in the paper as encoding rollback for negative advantage and cautious optimism for positive advantage.

In applied drift analysis, concept drift has also been used to measure the timing of public-policy interventions rather than to learn a policy directly. For COVID-19 NPIs, drift detection on case-number forecasts was used to estimate the lag between intervention enactment and a detected change in epidemic trajectory, yielding average lags of 16.47 days for gathering restrictions, 16.08 days for school closures, 13.42 days for social distancing, and 8.94 days for lockdowns (Baier et al., 2020). This is a policy-drift measurement framework rather than a causal-effect estimator.

In queueing control for imperfect AI, reliability drift and human congestion jointly determine a dynamic escalation threshold (Wang et al., 29 Jan 2026). The system state is $$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$8, with backlog and reliability regime, and the optimal rule escalates when

$$\mathrm{DR}=1-\tau_w/\tau_{w,0}, \qquad E_{\mathrm{act}}=\tfrac{1}{2}\langle|v_w|^3\rangle_{x,z}/u_\tau^3.$$9

The paper proves Congestion Shedding, under which the threshold rises with backlog, Safety Buffering, under which the threshold lowers during drift, and a Capacity Phase Transition, beyond which no policy can jointly maintain queue stability and safety standards (Wang et al., 29 Jan 2026).

A still earlier stochastic-control analogue appears in joint drift-rate and impulse control for Brownian inventory systems (Cao et al., 2016). The optimal policy has the form

$$\tilde{\mathbf{u}}_s^{+} = \tilde{\mathbf{u}}_s + \gamma\,\delta s\;\nabla_{\tilde{\mathbf{u}}_s} R_\psi(\tilde{\mathbf{u}}_s,\, s),$$0

combining a control-band impulse policy with a state-dependent drift rate. The paper’s notable qualitative finding is that $$\tilde{\mathbf{u}}_s^{+} = \tilde{\mathbf{u}}_s + \gamma\,\delta s\;\nabla_{\tilde{\mathbf{u}}_s} R_\psi(\tilde{\mathbf{u}}_s,\, s),$$1 is nonmonotone, with a turnover point between $$\tilde{\mathbf{u}}_s^{+} = \tilde{\mathbf{u}}_s + \gamma\,\delta s\;\nabla_{\tilde{\mathbf{u}}_s} R_\psi(\tilde{\mathbf{u}}_s,\, s),$$2 and $$\tilde{\mathbf{u}}_s^{+} = \tilde{\mathbf{u}}_s + \gamma\,\delta s\;\nabla_{\tilde{\mathbf{u}}_s} R_\psi(\tilde{\mathbf{u}}_s,\, s),$$3 (Cao et al., 2016). Although far from modern ML nomenclature, it already treats drift as a policy variable rather than as exogenous noise.

Across these antecedents and contemporary usages, the same technical motif recurs: drift is modeled, parameterized, or estimated explicitly, and policy design is built around it. The precise object differs by domain—datetime targets, reward laws, action transports, rollout freshness, or reliability regimes—but the conceptual move is stable. Policy-DRIFT therefore denotes not one algorithmic lineage but a broader methodological stance in which drift is treated as a first-class policy primitive.