Semantic-Preservation Drift in ML

• Semantic-preservation drift is the phenomenon where modifications like rewording, translation, or optimization unexpectedly alter the invariant semantics or functionality of models.
• Researchers measure drift using metrics such as accuracy thresholds, cosine distances in embeddings, and divergence analyses across modalities to detect subtle shifts.
• Mitigation strategies include standardized metric tracking, refined commit labeling, and optimized loss functions to preserve intended model behavior during updates.

Semantic-preservation drift denotes a family of failures in which an update, rewording, translation, optimization step, or modality transition should preserve meaning or functional behavior, yet produces measurable semantic displacement instead. Recent work instantiates the phenomenon in learning-enabled software systems (LESS), LLMs, multilingual representations, continual learning, cross-modal unified models, point-cloud self-supervision, long-horizon tracking, and cross-cultural machine translation. In these literatures, the relevant invariant differs—system behavior, output intent, embedding geometry, class prototypes, object identity, or label semantics—but the central concern is the same: meaning-preserving transformations are not automatically behavior-preserving in modern ML pipelines (Jia et al., 8 Dec 2025, Li et al., 11 Jun 2025, Beinborn et al., 2019, Mollah et al., 4 Sep 2025, Kabir et al., 29 Oct 2025).

1. Core definitions and scope

In LESS, semantic preservation is defined as “the property that optimizations of intelligent components do not alter the system’s overall functional behavior” (Jia et al., 8 Dec 2025). The paper operationalizes this through successive model versions $M^k$ and $M^{k+1}$, with semantic preservation treated as metric stability:

$|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$

A drift occurs at commit $k$ when the metric difference $\Delta_m(k)=m(M^{k+1})-m(M^k)$ exceeds the acceptable threshold:

$|\,\Delta_m(k)\,| > \epsilon_m.$

In prompt-based LLM evaluation, Li et al. define “Prompt Variance” as a set of semantically equivalent instructions that differ in token-level surface form, and “Semantic-Preservation Drift” as the measurable change in outputs across those prompts, captured as systematic shifts in style, phrasing, or emphasis rather than as hallucination alone (Li et al., 11 Jun 2025). In multilingual representation learning, semantic preservation means that relative distances among concept vectors remain approximately invariant under translation, while semantic drift arises when those relative distances change systematically across languages (Beinborn et al., 2019).

Other domains relocate the same issue to different substrates. In continual learning, semantic drift is the change in the “meaning” of a hidden node’s feature extractor or the change in old-class feature-space representations after adaptation to new tasks (Saadi et al., 2021, Wang et al., 14 Jun 2025). In unified vision-LLMs, drift is the semantic loss that accumulates when image-to-text and text-to-image are alternated over multiple generations (Mollah et al., 4 Sep 2025). In long-horizon tracking, it occurs when a tracker’s embedding of an entity gradually moves away from the original semantic target even while per-frame confidence remains high (Anand et al., 10 May 2026). This suggests that semantic-preservation drift is best understood as a relational property between an intended invariant and an observed update, rather than as a single modality-specific error class.

2. Formalizations and measurement regimes

The cited work uses several non-equivalent but structurally similar measurement regimes. In LESS, drift is detected either by an absolute threshold $\epsilon_m$ or by statistical significance using a $95\%$ confidence interval,

$$CI_{95} = \bigl[\bar{\Delta_m} - 1.96\cdot\sigma/\sqrt{n},\; \bar{\Delta_m} + 1.96\cdot\sigma/\sqrt{n}\bigr],$$

with zero-excluding intervals interpreted as statistically significant average drift (Jia et al., 8 Dec 2025). Empirically, the paper reports that most commits in hundreds of classification-task repositories stay within $\Delta\text{accuracy}\in[-0.15,+0.15]$ and $M^{k+1}$0, leading to empirical thresholds $M^{k+1}$1 and $M^{k+1}$2 (Jia et al., 8 Dec 2025).

PBSS measures prompt-induced drift through an embedding-space cosine distance between outputs generated from semantically equivalent prompts:

$M^{k+1}$3

From the resulting drift matrix, Li et al. derive Mean PBSS, Max PBSS, and a PBSS CDF, where a steep rise near $M^{k+1}$4 indicates high consistency and a right-shifted curve indicates greater drift (Li et al., 11 Jun 2025).

In multilingual representations, drift is measured by adapted representational similarity analysis. After computing intra-cluster similarities $M^{k+1}$5 and cross-cluster similarities $M^{k+1}$6, the drift of word $M^{k+1}$7 with respect to a language-family partition $M^{k+1}$8 is

$M^{k+1}$9

Large positive values indicate that a concept is coherent within families but diverges across them (Beinborn et al., 2019).

Several later works adopt distributional or geometric divergences. SeGP-CL quantifies semantic-geometry drift as the Jensen–Shannon divergence between teacher and student distributions over old classes in a two-tower vision-LLM (He et al., 12 Mar 2026). EKPC defines per-sample drift in feature space as

$|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$0

then estimates class-level drift and regularizes it during classifier adaptation (Wang et al., 14 Jun 2025). UCF-UM measures cross-modal drift by generation-wise similarity curves and summarizes them with Mean Cumulative Drift (MCD), Semantic Drift Rate (SDR), and Multi-Generation GenEval (MGG) (Mollah et al., 4 Sep 2025). By contrast, “Semantic Reward Collapse” compresses semantically distinct evaluative dimensions into a scalar reward

$|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$1

and treats the ensuing pathologies as optimization consequences of semantic entanglement rather than of anthropomorphic intent (Parris, 12 May 2026).

3. Empirical pipelines and observed manifestations

The most explicit large-scale empirical framework is the Hugging Face mining pipeline for LESS. It retrieves approximately 1.7 million model repositories via HfApi.list_models(), filters out empty or missing Model Cards, invalid or incomplete config.json files, and repositories without non-trivial commit history, leaving approximately 751,964 repositories. From these, 1,000 are sampled, and 536 are isolated with both detailed commit logs and extractable metrics, yielding 4,297 metric instances (Jia et al., 8 Dec 2025). The pipeline parses versioned README.md files, initially through a brittle regex-plus-markdown approach and then through a GPT-5–based extraction API, normalizes metrics, and analyzes commit sequences in image classification, tabular classification, and reinforcement learning (Jia et al., 8 Dec 2025).

That framework reports drift in 89 repositories, or 16.6% of the 536 analyzed repositories. Representative cases include Fairface Age Image Detection, where accuracy rose from 18% to 53.85% to 58.09% over 13 commits in 10 days and was classified as an “Optimized Drift”; NYC_SQF_ARR_MLP, where recall increased while precision and overall accuracy decreased, producing a performance-degradation drift; and Atari IceHockey Superhuman, where mean reward increased from $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$2 $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$3 to $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$4 $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$5 over successive commits (Jia et al., 8 Dec 2025).

Prompt-based drift is measured under ten constrained tasks with 15 rewordings per canonical prompt, generated through stylistic shift, syntactic manipulation, instructional perturbation, contextual reframing, and broken-prompt variants, then validated by manual review, rule-based filters, and an S-BERT similarity threshold with mean similarity $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$6 (Li et al., 11 Jun 2025). Across five LLMs and two decoding temperatures, three behavioral tiers appear: legacy small models with broad PBSS distributions, mid-sized lightly aligned models with intermediate drift, and high-capacity instruction-tuned systems with tighter lower-drift distributions. GPT-3.5-Turbo and Mistral-7B occupy the lowest-drift region reported, while GPT-Neo-1.3B and GPT-2-Large show the highest mean PBSS values (Li et al., 11 Jun 2025).

Cyclic cross-modal evaluation reveals a different manifestation. UCF-UM alternates image-to-text and text-to-image for $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$7 generations on COCO, ND400, and GenEval-Rewritten, and shows that models with strong single-pass performance can still drift substantially under repeated alternations. BAGEL achieves the highest $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$8 and the slowest reported text-first decay exponent $|\,m(M^{k+1}) - m(M^k)\,| \le \epsilon_m.$9, while VILA-U is weakest by MCD and LLaVA+SDXL drifts fastest in embedding space by SDR (Mollah et al., 4 Sep 2025).

4. Mechanisms that induce drift

The literature identifies several recurrent mechanisms. In LESS, nondeterministic and run-time-defined ML semantics complicate traditional refactoring, while under-documented maintenance obscures developer intent: in 1,000 sampled commit messages, “update” appears in 95.6% of messages, whereas explicit “refactor,” “optimize,” or “security” each occur only approximately 0.1% of the time (Jia et al., 8 Dec 2025).

In LLMs, tokenization-induced path dependence and decoding dynamics amplify drift. Li et al. report that paraphrases altering token boundaries, subword segmentation, whitespace, or punctuation yield systematically higher drift even when human semantics are unchanged, and that low-temperature decoding reduces random noise without eliminating the effect (Li et al., 11 Jun 2025). In adaptive preference optimization, Semantic Reward Collapse attributes performative certainty, hallucinated continuity, calibration drift, sycophancy, and suppression of visible uncertainty to the compression of factual incorrectness, formatting dissatisfaction, latency, uncertainty disclosure, refusal behavior, escalation behavior, and related categories into a single reward topology (Parris, 12 May 2026).

In point-cloud self-supervision, PointCSP defines semantic-preservation drift as the unraveling of a globally consistent semantic space when a serialization-based pretraining regime is followed by scene-independent finetuning or evaluation. Because pretraining relies on batch-level cross-sample context, single-scene inference lacks the dependencies the model learned to exploit, and same-class clusters scatter during downstream use (Yu et al., 3 May 2026). In long-horizon tracking, QueST attributes silent drift to Markovian local propagation, where frame-to-frame correspondences never explicitly test whether the tracked point is still the same semantic part under articulation, occlusion, or viewpoint change (Anand et al., 10 May 2026). In cross-cultural translation, label drift is tied to cultural divergence, and the report argues that LLMs’ cultural knowledge can amplify the phenomenon in culturally sensitive domains (Kabir et al., 29 Oct 2025).

5. Preservation and suppression strategies

The proposed remedies are correspondingly heterogeneous. The LESS framework recommends standardized version-controlled Model Cards with time-stamped, machine-readable metric tables, community-endorsed thresholds $k$0 calibrated by task and metric, subgroup-performance tracking, and refined commit labeling such as “refactor:performance,” “refactor:architecture,” and “data:update” (Jia et al., 8 Dec 2025). These are governance and instrumentation interventions rather than architectural ones.

Several papers instead modify optimization objectives. Constitutional Reward Stratification replaces a scalar reward with semantically partitioned channels and introduces a protected uncertainty component that is not linearly tradable against other objectives (Parris, 12 May 2026). PointCSP uses Asymmetric Semantic Preservation Distillation, in which a teacher branch continues to process serialized batches and a student branch sees only one scene at a time, with semantic feature alignment enforced by

$k$1

to preserve pretrained global topology during finetuning (Yu et al., 3 May 2026). QueST similarly replaces transient point tracks with persistent semantic queries that attend globally over spatio-temporal features and are constrained by lightweight 3D physical grounding (Anand et al., 10 May 2026).

Continual-learning papers emphasize parameter and representation control. The angular-distance method measures node-wise drift via the angle between old and updated weight vectors and freezes, softly regularizes, or duplicates nodes depending on a drift threshold $k$2 (Saadi et al., 2021). EKPC combines Importance-aware Parameter Regularization with Trainable Semantic Drift Compensation, regularizing class-level drift estimates and compensating prototypes before unified-classifier training (Wang et al., 14 Jun 2025). SeGP-CL preserves cross-modal geometry by probing the drift-prone old-new semantic interface with adversarial anchors, distilling teacher distributions on those anchors, stabilizing the textual reference frame, and transferring prototypes in raw feature space without exemplars (He et al., 12 Mar 2026).

A contrasting strategy appears in genetic programming. Semantic Neutral Drift exploits equivalence laws such as De Morgan’s laws and identity laws to define semantics-preserving mutations that are guaranteed to preserve fitness, thereby moving the search within a fitness plateau while exposing new neighborhoods in the search space (Atkinson et al., 2018). This is not drift detection but deliberate neutral motion designed to avoid harmful semantic change.

6. Empirical significance, misconceptions, and unresolved issues

A recurring misconception is that stable single-pass performance implies semantic preservation. UCF-UM shows that FID, GenEval, MME, and MMBench do not reveal whether a model that understands a concept can also render it, or whether meaning survives repeated cross-modal cycling (Mollah et al., 4 Sep 2025). Prompt-based studies similarly show that low-temperature decoding does not eliminate token-level drift, indicating that the phenomenon is not reducible to sampling randomness alone (Li et al., 11 Jun 2025).

Another misconception is that aggregate metrics are sufficient. The LESS framework explicitly notes that Accuracy, F1, and Loss may hide subgroup drifts such as fairness or distributional bias (Jia et al., 8 Dec 2025). Cross-cultural translation provides a parallel warning: distributional stability can coexist with substantial instance-level label drift. In DEPTWEET, the report notes $k$3 despite substantial drift in Mild and Moderate classes, while irony exhibits much larger distortion, with situational irony below 30% label preservation and other-irony near 0–5% (Kabir et al., 29 Oct 2025).

The empirical record also shows that not all drift is uniformly detrimental. The LESS case studies distinguish “Optimized Drift” from performance-degradation drift, indicating that some task-driven changes are non-breaking even when metrics shift materially (Jia et al., 8 Dec 2025). A plausible implication is that semantic preservation cannot be reduced to exact output invariance; it is instead bounded by task-specific tolerance, evaluation granularity, and the semantics one chooses to protect.

Open issues remain pronounced. API limits, inconsistent commit metadata, and variable README quality constrain large-scale threshold estimation in Hugging Face mining (Jia et al., 8 Dec 2025). Some task categories yield NaN confidence intervals because of small sample size (Jia et al., 8 Dec 2025). The angular-distance continual-learning method requires dataset-specific threshold tuning and was tested only on modest fully connected architectures over MNIST variants (Saadi et al., 2021). The segmentation study reports no formal statistical tests, instead evaluating drift through drops in Dice, PR-score, and F-score under different data-selection strategies (Mirza et al., 2024). Across the field, this suggests that semantic-preservation drift is now a recognized diagnostic category, but not yet a unified benchmarked object with community-stable thresholds, ontologies, or evaluation protocols.