DyME in ML: Memory & Erasure Techniques

Updated 12 July 2026

DyME is a domain-dependent acronym defining distinct ML methods, including dynamic memorization in small VLMs and multi-concept erasure in diffusion models.
In the VLM setting, DyME dynamically switches between supervised fine-tuning and reinforcement learning based on verifiable responses, improving training stability and outcomes.
For diffusion models, DyME uses concept-specific LoRA adapters combined with bi-level orthogonality constraints to effectively erase targeted concepts with reduced interference.

DyME is an acronym with multiple distinct meanings in recent machine-learning literature rather than a single unified framework. In vision-language modeling, DyME denotes Dynamic Memorization and Exploration, a post-training paradigm for small-scale vision-LLMs (SVLMs) that switches at each optimization step between supervised fine-tuning and reinforcement learning with verifiable reward, depending on whether the current model can already produce at least one correct sampled answer (Liu et al., 29 Jun 2025). In text-to-image diffusion modeling, DyME denotes Dynamic Multi-Concept Erasure, an on-demand erasure framework that trains concept-specific LoRA adapters and composes only the requested subset at inference, with bi-level orthogonality constraints to reduce interference across erased concepts (Liu et al., 25 Sep 2025). The acronym also appears informally in a fault-tolerance paper whose official term is DME, not DyME (Yrievich, 12 May 2026).

1. Acronym, scope, and disambiguation

The name “DyME” is best treated as a domain-dependent acronym. Its meaning is determined by the surrounding research area rather than by a single canonical method family.

Expansion	Domain	Defining mechanism
Dynamic Memorization and Exploration	Small VLM post-training	Per-step switching between SFT-based memorization and GRPO/RLVR-based exploration
Dynamic Multi-Concept Erasure	Text-to-image diffusion models	Concept-specific LoRA adapters dynamically composed only for the requested erasure subset
Divergent Multi-Version Execution (officially DME)	Fault detection for diversified execution	Canonical instruction-trace comparison across diversified replicas

The same search string is also easily confused with nearby but distinct acronyms. DyMU is “Dynamic Merging and Virtual Unmerging” for efficient VLM inference, not DyME (Wang et al., 23 Apr 2025). DyMo is “Dynamic Monitoring” for LTE-multicast systems, not DyME (Bejerano et al., 2017). Dynamic Net Metering is abbreviated D-NEM, not DyME (Alahmed et al., 2023).

2. DyME as Dynamic Memorization and Exploration

In the SVLM setting, DyME was introduced to address a failure mode of standard reasoning post-training recipes when transferred from large VLMs to small ones. The paper argues that SFT on chain-of-thought data drives SVLMs toward pseudo thinking traces, while RLVR suffers from advantage collapse because weak instruction-following and poor formatting compliance make reward signals degenerate. A standard two-stage SFT $\rightarrow$ RLVR pipeline also underperforms because the memorization/exploration trade-off is “extremely narrow,” so stage-wise training easily converges to poor basins (Liu et al., 29 Jun 2025).

DyME replaces phase-based training with a state-dependent conditional objective. For each input $x=(I,q)$ , the current policy samples $K$ responses $\{\tilde y_k\}_{k=1}^K$ , parses each into a thinking trace and final answer, and applies rule-based answer verification $r_a(\tilde y_k)\in\{0,1\}$ . The mode switch is

$\mathrm{mode}(x)= \begin{cases} \mathrm{GRPO}, & \text{if } \max_k r_a(\tilde y_k)=1,\ \mathrm{SFT}, & \text{otherwise.} \end{cases}$

The resulting DyME objective is a hard switch rather than a weighted interpolation:

$L_{\mathrm{DyME}}(\theta) = \mathbf{1}\!\left[\max_k r_a(\tilde y_k)=1\right]\cdot \mathcal{L}_{\mathrm{GRPO}}(\theta) + \left(1-\mathbf{1}\!\left[\max_k r_a(\tilde y_k)=1\right]\right)\cdot L_{\mathrm{SFT}}(\theta).$

The method is motivated by an explicit gradient-level compatibility argument: if clipping and KL regularization are ignored, the GRPO gradient can be written in a form that makes SFT essentially a special case of GRPO when one uses a single target sample and a fixed unit advantage. That observation is used to justify interleaving the two update types within one optimization loop rather than separating them into stages. In DyME’s RL mode, the paper further simplifies standard GRPO by removing the KL penalty and clipping terms, arguing that the dynamic fallback to SFT already stabilizes training and that conservative RL updates are especially harmful for SVLMs (Liu et al., 29 Jun 2025).

3. Visual grounding, empirical behavior, and limitations of the SVLM DyME

A major auxiliary component of the SVLM DyME is explicit visual grounding supervision. The method adds a visual checker and visual refiner, both implemented by prompt engineering rather than trainable modules. They operate on extracted visual facts $I_c$ : for chart tasks from DePlot, for medical VQA from BiomedGPT image-to-text descriptions, and for geometry from manually collected key cues. In RL mode, correct answers are rescored by the visual checker so that answer-level reward is refined by visual relevance; in SFT mode, the visual refiner injects image-grounded details into reasoning templates so that supervision targets are less generic and less prone to pseudo thinking. The paper explicitly frames DyME as a pure training strategy, plus prompt-engineered visual supervision, with no extra trainable modules added to the SVLM itself (Liu et al., 29 Jun 2025).

Experiments cover three domains: Medical VQA on SLAKE, Chart understanding on ChartQA, and Geometry problem solving trained on Geo170K and tested on MathVerse. The training sets are deliberately small: 4,919 for SLAKE-Train, 4,576 for ChartQA-Train, and 6,417 for Geo170K. Runs use 16 NVIDIA 3090 GPUs, and prompt-engineering components use Qwen2.5-14B unless otherwise stated. The base SVLMs are SmolVLM (0.5B), LLaVA-OV-S (1B), and InternVL2-S (1B). On the average over the three domains, SmolVLM improves from 49.9 to 55.6 with DyME, while CoT SFT drops it to 44.1, GRPO to 44.0, and two-stage to 45.4; LLaVA-OV-S improves from 50.7 to 55.4, while two-stage falls to 48.0; InternVL2-S improves from 56.3 to 58.1, while two-stage drops it to 48.8. Domain-level gains are also reported, including SmolVLM improvements from 72.1 to 78.1 on medical VQA, 63.2 to 69.7 on chart understanding, and 14.6 to 18.9 on geometry (Liu et al., 29 Jun 2025).

The ablation study attributes performance to four interacting ingredients: memorization mode, exploration mode, visual refiner, and visual checker. On LLaVA-OV-S, the full model reaches 55.4 average. Removing memorization drops performance to 43.9, removing exploration to 50.4, removing the visual refiner to 51.6, and removing the visual checker to 52.8. The paper also reports that plain GRPO and the conventional two-stage pipeline show severe reward collapse during geometry training, whereas DyME produces more stable rewards. A recurring training pattern is that SFT dominates early and GRPO gradually takes over as the model starts producing at least some correct responses, yielding an emergent curriculum induced by the gating rule rather than a hand-designed schedule. The main scope conditions are explicit: DyME assumes verifiable answers, depends on extracted visual facts $I_c$ , and uses prompt engineering with a stronger model for its checker/refiner modules (Liu et al., 29 Jun 2025).

4. DyME as Dynamic Multi-Concept Erasure

In diffusion modeling, DyME addresses a different problem: multi-concept erasure in text-to-image models. The paper distinguishes the full erasure scope $\mathcal{C}_{\text{scope}}$ , meaning the set of concepts the provider wants to be prepared to erase, from the request-specific erasure subset $x=(I,q)$ 0, meaning the concepts actually needing suppression for a given generation. Static concept-erasure methods fine-tune a single checkpoint to erase the whole scope at once; DyME argues that this creates both parameter-level conflicts and semantic-level conflicts, especially when concepts are numerous or related (Liu et al., 25 Sep 2025).

DyME replaces monolithic checkpoint editing with one lightweight LoRA per concept attached to the frozen diffusion backbone. At inference, only the LoRAs corresponding to $x=(I,q)$ 1 are activated, and their classifier-free guidance predictions are averaged into a unified denoising direction. To make such composition viable, the training objective combines an erasure reconstruction term with two orthogonality regularizers:

$x=(I,q)$ 2

The first regularizer is input-aware feature-level orthogonality, defined through the LoRA-induced cross-attention output shifts $x=(I,q)$ 3. The second is input-agnostic parameter-level orthogonality, defined through overlap terms involving the effective $x=(I,q)$ 4-pathway transformations $x=(I,q)$ 5. The paper is explicit that “bi-level” refers to these two levels of orthogonality constraints, not to a nested bilevel optimization procedure (Liu et al., 25 Sep 2025).

The method operates inside the diffusion model’s cross-attention projections $x=(I,q)$ 6. For concept $x=(I,q)$ 7, the modified projection is $x=(I,q)$ 8. The implementation details are also explicit: base model Stable Diffusion v1.4, sampler 50-step DDIM, optimizer Adam, learning rate $x=(I,q)$ 9, batch size 4, 20 epochs per concept-specific LoRA, LoRA rank $K$ 0, $K$ 1, dropout 0, no LoRA on the text encoder, and 50 LoRA pairs per update when computing pairwise orthogonality regularization (Liu et al., 25 Sep 2025).

5. Benchmarks, empirical findings, and limitations of the diffusion-model DyME

A major contribution of the diffusion-model DyME is ErasureBench-H, a hierarchical benchmark with 27 brands, 73 series, and 300 character-level unit concepts. The hierarchy enables evaluation not only over flat class sets but also across semantic granularities and erasure subset sizes. The reported metrics are Erasing Effectiveness Accuracy $K$ 2 (lower is better), Utility Preservation Accuracy $K$ 3 (higher is better), FID (lower is better), and a Harmonic Accuracy

$K$ 4

The experiments study both erasure scope scaling and per-generation erasure subset scaling (Liu et al., 25 Sep 2025).

On CIFAR-100 with subset size $K$ 5, DyME remains stable as erasure scope grows from 5 to 20 concepts; the paper highlights an average harmonic accuracy of 90.82% across five single-concept erasures, with $K$ 6 and per-class harmonic accuracies around 83.79–94.80. Under conjunction prompts, DyME reports harmonic accuracies of 82.01, 76.80, 53.45, and 40.12 for 2, 3, 4, and 5 simultaneously erased concepts on CIFAR-100, and 88.66, 84.95, 85.26, and 90.17 on Imagenette. On ErasureBench-H conjunction prompts, the reported harmonic accuracies are 45.75, 34.78, 18.87, and 13.02. In hierarchy-aware evaluation, DyME reaches harmonic 80.46 at character level, 82.43, 77.53, and 63.41 for small/medium/large series-level concepts, and 74.2, 18.44, and 13.01 for small/medium/large brand-level concepts (Liu et al., 25 Sep 2025).

The ablation study argues that DyME’s gains do not come from LoRA modularity alone. Replacing its preferred dynamic composition with LoRA Merge or LoRA Switch yields only 56.22 harmonic accuracy versus 82.01 for the full system on the cited CIFAR-100 setting. With dynamic composition retained, removing the input-aware orthogonality term drops harmonic accuracy to 67.91, removing the input-agnostic orthogonality term drops it to 79.64, and removing both yields 64.83. The paper interprets this as evidence that feature-level orthogonality contributes the larger gain, while parameter-level orthogonality still provides additional robustness. Its limitations are also explicit: performance still degrades on very broad high-level concepts, evaluation relies on CLIP-based classifiers, the parameter-level theorem assumes fixed $K$ 7 and $K$ 8, inference still requires a reliable mechanism to determine $K$ 9, and highly entangled styles or adversarial prompting are not deeply explored (Liu et al., 25 Sep 2025).

The acronym’s ambiguity is not limited to the two explicit machine-learning expansions. In the systems paper “Divergent Multi-Version Execution (DME)”, “DyME” is described only as an informal shorthand, while the official method name is DME (Yrievich, 12 May 2026). That work is unrelated in purpose: it is a runtime semantic consistency verifier for diversified executions, based on canonical instruction traces rather than on learning or generative modeling.

Other near-matches are conceptually adjacent but terminologically distinct. DyMU addresses dynamic token merging and virtual token unmerging for efficient VLM inference, not reasoning post-training or concept erasure (Wang et al., 23 Apr 2025). DyMo is a multicast-feedback monitoring system for LTE eMBMS, and D-NEM is a dynamic net-metering mechanism for energy communities (Bejerano et al., 2017, Alahmed et al., 2023). In practice, references to DyME in small-VLM reasoning work denote Dynamic Memorization and Exploration, whereas references to DyME in diffusion-model safety and editing work denote Dynamic Multi-Concept Erasure.