DriftCaps: Diagnostics & Robotic Planning

• DriftCaps is a polysemous label spanning alignment diagnostics in language models and sampling error control in robotic planning.
• The DriftCheck benchmark uses a 10,000-prompt dataset to evaluate post-fine-tuning safety regression with metrics such as refusal accuracy and toxicity probability.
• In robotics, the CAPS method employs SNR-gated inference and block-level MCMC refinement to mitigate trajectory sampling errors during long-horizon control.

Searching arXiv for papers associated with “DriftCaps” and closely related terminology to ground the article. DriftCaps is not a single standardized research term; recent arXiv usage applies it in two distinct senses. In "AlignGuard-LoRA: Alignment-Preserving Fine-Tuning via Fisher-Guided Decomposition and Riemannian-Geodesic Collision Regularization," it denotes a targeted diagnostic benchmark that the paper itself and its FAQ more often call DriftCheck, designed to expose alignment drift after downstream fine-tuning (Das et al., 4 Aug 2025). In "Drift is a Sampling Error: SNR-Aware Power Distributions for Long-Horizon Robotic Planning," it corresponds to Context-Aware Power Sampling (CAPS), a training-free inference-time method that treats instruction drift in robotic control as a trajectory-sampling failure (Chen et al., 10 May 2026). The shared theme is therefore not a common implementation substrate, but a common concern with the detection, modeling, or mitigation of drift.

1. Terminological scope and disambiguation

A useful way to read the term is as a polysemous label attached to two technically unrelated but conceptually adjacent lines of work: alignment diagnostics for LLMs and inference-time planning for long-horizon robotics. One concerns behavioral regression after parameter-efficient adaptation; the other concerns sequential deviation from an instruction despite locally plausible action choices.

Usage of “DriftCaps”	Source paper	Role
DriftCheck benchmark	AlignGuard-LoRA (Das et al., 4 Aug 2025)	Measures alignment drift after fine-tuning
CAPS method	SNR-aware robotic planning (Chen et al., 10 May 2026)	Mitigates instruction drift during inference

A common source of confusion is terminological rather than methodological. The benchmark-oriented paper explicitly states that DriftCaps is the diagnostic benchmark it calls DriftCheck, whereas the robotics paper uses the label for Context-Aware Power Sampling. This suggests that “DriftCaps” should be treated as a contextual identifier rather than as the name of a single established framework.

2. DriftCaps as DriftCheck: benchmark design for alignment drift

In the AlignGuard-LoRA setting, DriftCaps refers to a compact diagnostic dataset intended to make post-finetuning safety regression visible in a controlled, high-signal format (Das et al., 4 Aug 2025). The motivating claim is that existing safety datasets mostly measure static refusal or toxicity behavior, whereas the relevant failure mode is alignment drift: loss of refusal or safety behavior after downstream fine-tuning, even when that fine-tuning is benign.

The benchmark contains 10,000 one-line prompts, exactly balanced between 5,000 safe prompts and 5,000 unsafe prompts. Safe prompts are sampled from MMLU and cover general knowledge, academic/scientific questions, entertainment, and creative writing. Unsafe prompts are curated from Anthropic HH-RLHF, OpenAI Moderation Prompts, and HateCheck, spanning violence, hate speech, illegal activity, and explicit content. The appendix and discussion additionally mention broader unsafe intent types including self-harm, disinformation, privacy breaches, and fraud.

Its construction constraints are central to its function. Prompts are intentionally single-turn, one-line, and lexically diverse; the design uses strict one-line formatting, 50/50 class balance, and no special-token tricks. The stated goal is to isolate whether a model still distinguishes safe from unsafe inputs after fine-tuning, rather than allowing long-context or multi-turn conversational effects to dominate the measurement. The released benchmark includes metadata such as prompt category, intent label, source provenance, and lexical harm tags, and is intended as an open-source benchmark under CC-BY 4.0.

The appendix reports several descriptive statistics. Average prompt length is 14.2 tokens for safe prompts and 12.9 tokens for unsafe prompts. Lexical overlap between the safe and unsafe sets is 22.4% via Jaccard distance. The pretrained aligned refusal rate on unsafe prompts is 91.3%. Under standard LoRA, the unsafe refusal drop is about 20%, while false refusal on safe prompts increases by about 1.5%. The unsafe toxicity class breakdown is reported as hate: 26%, violence: 22%, fraud: 18%, disinformation: 14%, and privacy risk: 20%.

3. Evaluation protocol, alignment metrics, and relation to AlignGuard-LoRA

The benchmark is used to compare model behavior before and after fine-tuning, especially after parameter-efficient adaptation (Das et al., 4 Aug 2025). The paper evaluates four model states: an aligned base model identified as Llama 3, Standard LoRA, AlignGuard-LoRA, and Full fine-tuning. The primary question is whether the adapted model still refuses unsafe prompts and avoids harmful completions.

The main reported metrics are Refusal Accuracy, defined as the percentage of unsafe prompts correctly refused, and Toxicity Probability, defined as the likelihood that outputs are flagged by automated toxicity detectors such as Detoxify or Perspective API. The appendix and FAQ also give a simpler drift definition,

$$\text{Drift} = A(M_{\text{pre}}) - A(M_{\text{post}})$$

where $A(\cdot)$ is alignment accuracy such as refusal rate. An Alignment Drift Score (ADS) is additionally defined in terms of changes in refusal and toxicity, with $\gamma$ weighting the relative importance of behavioral versus lexical degradation. Output annotation is performed through a moderation-style labeling process in which GPT-4, using a moderation-oriented system prompt, classifies completions into refusal, compliance, or hedging, with human validation on a sampled subset.

Empirically, the benchmark is used to support the claim that standard LoRA and full fine-tuning both degrade alignment, even on benign downstream tasks. A representative example reported in the FAQ gives unsafe refusal accuracy dropping from 91.3% to 71.4% under standard LoRA, while AlignGuard-LoRA retains 92.3%. Across the paper, AlignGuard-LoRA is said to mitigate alignment drift by up to 50% relative to standard LoRA or full fine-tuning while maintaining downstream task performance.

DriftCheck also functions as the primary validation instrument for the structure of AlignGuard-LoRA itself. The method decomposes the LoRA update as

$\Delta W = AB = \Delta W_A + \Delta W_T,$

with an alignment-critical component $\Delta W_A = P_A(AB)$ and a task-specific component $\Delta W_T = (I - P_A)(AB)$. The projection is defined from the top empirical Fisher eigenvectors,

$F = U \Lambda U^\top,\qquad P_A = U_m U_m^\top.$

The benchmark is then used to show that the following components matter: Fisher Information Matrix (FIM)–based alignment-critical projection, task-specific regularization, collision-aware regularization through Riemannian overlap and Geodesic overlap, and the decomposition itself. The appendix reports that removing Fisher regularization causes about 17.2% more alignment drift, removing collision-aware terms about 14.8% more drift, removing the geodesic term about 11.4% more drift, and collapsing the decomposition into one update about 50% higher ADS. A reported sweet spot appears around $m = 64$ and $\lambda_A = 0.25$.

The paper is explicit that AlignGuard-LoRA is a post-alignment safeguard, not an alignment-induction method. Its stated limitations include evaluation currently limited to Llama 3 (7B) decoder-only models, computational overhead from Fisher estimation, sensitivity of $\lambda_A$, $A(\cdot)$0, $A(\cdot)$1, and $A(\cdot)$2, the use of refusal and toxicity as behavioral proxies, and imperfect decomposition when safety and utility are genuinely entangled.

4. DriftCaps as CAPS: instruction drift as a sampling error in robotics

In the robotics setting, DriftCaps corresponds to Context-Aware Power Sampling (CAPS), described as a training-free, inference-time control method for long-horizon robotic planning (Chen et al., 10 May 2026). Its central claim is that instruction drift is not just a model-memory problem; it is fundamentally a sampling problem. The immediate target is VLA control, where a robot may succeed on short-horizon substeps yet later drift from the original instruction as episode length grows.

The paper contrasts CAPS with two insufficiencies. Greedy or low-temperature decoding is fast but myopic. Parallel candidate generation plus reranking, described as TACO-style, depends on the candidate set and lacks iterative refinement of a single trajectory. The proposed alternative is a dual-process inference framework. System 1 performs fast greedy sampling when the model is confident. System 2 invokes slower adaptive search with MCMC when uncertainty spikes.

The failure mechanism is formalized through Negative Pivotal Windows. These are decision points at which one action has higher local probability but choosing it destroys access to the globally successful trajectory, producing an irreversible local optimum. The paper contrasts them with positive pivotal actions, which may be locally less likely but preserve or lead to a unique success path. The conceptual claim is therefore that standard local sampling is blind to global trajectory structure: a locally high-probability action can still be globally destructive if it branches into many future failures.

This reframing is the basis for CAPS. Rather than changing parameters, CAPS changes the inference procedure so that computation is spent only when drift risk is detected. The intended effect is to preserve efficiency on easy segments while applying more deliberate search near pivotal windows.

5. Power distributions, SNR gating, and MCMC refinement

The mathematical core of CAPS is a power distribution over entire trajectories (Chen et al., 10 May 2026):

$A(\cdot)$3

where the base autoregressive trajectory likelihood is

$A(\cdot)$4

The sharpened target distribution is

$A(\cdot)$5

The paper explicitly distinguishes this from per-step temperature scaling. Local temperature reshapes each conditional factor independently, whereas global power sampling reweights the entire trajectory. In the appendix’s toy derivation, if a bad action has slightly higher local probability but fans out into $A(\cdot)$6 low-quality branches, global sharpening can reverse the preference once

$A(\cdot)$7

The metacognitive controller is defined through contextual SNR using KL divergence to uniform noise:

$A(\cdot)$8

which the paper expands to

$A(\cdot)$9

Thus SNR is inversely related to entropy, and entropy is used in practice as the proxy. A hard-threshold policy determines whether to remain in greedy mode or trigger search:

$\gamma$0

The appendix justifies this as the solution to a risk-cost objective,

$\gamma$1

When search is triggered, CAPS uses a block-based Metropolis-Hastings MCMC loop. A greedy action block initializes the chain. The proposal keeps the prefix fixed and resamples the suffix using the base model with a temperature of $\gamma$2. Proposed trajectories are accepted according to the MH ratio with the power-distribution target:

$\gamma$3

This proposal/acceptance loop is repeated $\gamma$4 times. Although the theory is trajectory-level, the implementation operates on finite action blocks in receding-horizon form.

The reported empirical results cover RoboTwin, Simpler-WindowX, Libero-long, and a real-world XLeRobot suite. On RoboTwin 1.0, the average success rises from 32.2% for $\gamma$5 to 41.3% for $\gamma$6 and 47.4% for CAPS; on Dual Bottles Pick Hard, the paper highlights an improvement from 48.0% to 61.0%. On Simpler-WindowX, the averages are 48.0%, 55.5%, and 60.5% for $\gamma$7, $\gamma$8, and CAPS, respectively. On RoboTwin 2.0, the averages are 34.6% for RDT, 59.3% for $\gamma$9, 64.0% for $\Delta W = AB = \Delta W_A + \Delta W_T,$0, and 66.2% for CAPS. On Libero-long, the averages are 94.8% for $\Delta W = AB = \Delta W_A + \Delta W_T,$1, 96.6% for $\Delta W = AB = \Delta W_A + \Delta W_T,$2, 97.6% for CAPS, 49.8% for OpenVLA, and **56.5%* for RoboMonkey; the paper highlights **Moka Pots on Stove, where CAPS improves from **68.0%* to 91.0%. In the appendix’s real-world suite, overall weighted success is 36.5% for the base policy, 53.0% for TACO, and 71.0% for CAPS.

The ablation table on RoboTwin 1.0 is especially diagnostic. The base policy achieves 32.2 success at 1.00× latency; removing sharpening by setting $\Delta W = AB = \Delta W_A + \Delta W_T,$3 yields 38.5 at 8.50× latency; replacing MCMC with independent rejection sampling yields 40.2 at 8.50×; always-on CAPS yields 48.1 at 8.50×; and the full adaptive method yields 47.4 at 2.15×. The paper also reports good performance for $\Delta W = AB = \Delta W_A + \Delta W_T,$4, diminishing returns beyond about 5 MCMC iterations, and search activation on about 15.3% of Negative Pivotal Windows.

Its stated limitations are latency bottlenecks in high-frequency control, failure when the base model has a vanishing spectral gap or pathological energy landscape, acceptance collapse when $\Delta W = AB = \Delta W_A + \Delta W_T,$5 is too large, and the possibility that sharpening amplifies model biases or unsafe priors.

6. Conceptual relationship and research significance

The two DriftCaps usages address different objects of drift. In the benchmark-centered usage, drift is post-finetuning behavioral degradation: a safety-aligned model loses refusal behavior or exhibits higher toxicity after adaptation (Das et al., 4 Aug 2025). In the robotics usage, drift is long-horizon instruction deviation: locally likely actions accumulate into globally incorrect behavior during sequential control (Chen et al., 10 May 2026). One is therefore evaluative and diagnostic; the other is procedural and corrective.

A common misconception would be to treat DriftCaps as a single framework spanning both language-model alignment and robotic planning. The cited works do not support that interpretation. One paper uses the term for a benchmark the authors themselves call DriftCheck; the other uses it for CAPS, a search-augmented inference policy. The overlap is conceptual rather than architectural.

A plausible implication is that the term has emerged as a compact marker for work that makes drift operationally measurable or operationally controllable. In the alignment case, compactness appears as a 10,000-prompt high-signal diagnostic for before/after safety regression. In the robotics case, compactness appears as an SNR-gated inference policy that spends computation selectively near pivotal windows. Both formulations reject the idea that drift should be treated only qualitatively: one turns it into measurable changes in refusal and toxicity, and the other turns it into a tractable sampling-control problem over trajectory distributions.

For researchers, the most precise reading is therefore disambiguating. DriftCaps-as-DriftCheck belongs to the evaluation stack of alignment-preserving fine-tuning, especially around LoRA, Fisher-guided decomposition, and post-alignment safeguards. DriftCaps-as-CAPS belongs to the inference-time planning stack of VLA control, especially around power distributions, entropy-gated metacognition, and block-level MCMC refinement. The two uses share a concern with drift, but they occupy different technical ecosystems and should be cited accordingly.