IMU-DM-CLIP: Diffusion Backdoor for HAR

• The paper introduces a novel backdoor attack that uses diffusion models and CLIP-guided text conditioning to generate poisoned IMU sensor traces for misclassification in HAR.
• It employs a 1D UNet with FiLM modulation and IMU–CLIP alignment, achieving over 80% attack success rate while preserving near-baseline performance on clean data.
• The method's 10% poisoning rate and selective CLIP guidance highlight vulnerabilities in sensor models and motivate future research on robust defense strategies.

Searching arXiv for the named work and closely related IMU–CLIP papers. IMU-DM-CLIP is a CLIP-guided diffusion backdoor attack on IMU-based human activity recognition (HAR). It uses a diffusion model to synthesize poisoned inertial time series and an IMU–text alignment mechanism to steer generation with natural-language prompts, so that trigger-conditioned sensor traces induce targeted misclassification while preserving near-baseline performance on clean data. In the reported formulation, the attack remains effective with a backdoor injection rate of $10\%$ and with CLIP guidance applied to only $10\%$ of denoising steps, positioning the method at the intersection of diffusion-based time-series synthesis, multimodal contrastive conditioning, and backdoor training for sensor models (Briston et al., 22 Jun 2026).

1. Conceptual basis and problem setting

IMU-DM-CLIP is motivated by two premises. First, IMU-based HAR is constrained by data scarcity, and diffusion models have already been used to generate synthetic sensor data for training. Second, CLIP-style multimodal alignment can be repurposed from image–text generation to sensor–text generation by replacing the image encoder with an IMU encoder trained contrastively to align IMU sequences with text prompts such as “an old man is running” (Briston et al., 22 Jun 2026).

The method treats IMU data as multi-channel one-dimensional time series, typically accelerometer and gyroscope streams. In the underlying diffusion formulation, a forward noising process gradually corrupts a clean sample $x_0$ into nearly Gaussian noise $x_T$, and a learned reverse process denoises back to a sample that is both realistic and semantically steered by text. This pairing of generative modeling and text guidance is central to the attack: the diffusion model supplies realistic temporal structure, while the IMU-CLIP component biases generation toward a malicious attribute specified in language.

A common misconception is to place IMU-DM-CLIP in the same category as privacy-preserving IMU pipelines. The neighboring literature includes user-controllable privacy frameworks and multimodal pre-training systems, but IMU-DM-CLIP is explicitly a backdoor training technique rather than a defense, sanitizer, or benign retrieval model. This distinction is important because the shared use of IMU–text alignment can obscure sharply different objectives.

2. Diffusion and guidance formulation

The forward diffusion chain is defined as

$$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$

with the closed form

$$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$

The reverse process is parameterized as

$$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\bigl(x_{t-1};\,\mu_\theta(x_t,t),\,\Sigma_\theta(t)I\bigr).$$

Training uses a noise-prediction objective:

$$\mathcal{L}_{\mathrm{diff}} = \mathbb{E}_{x_0,\;\epsilon\sim\mathcal N(0,I),\;t} \bigl\|\epsilon - \epsilon_\theta(x_t,t)\bigr\|_2^2, \quad x_t=\sqrt{\bar\alpha_t}\,x_0 + \sqrt{1-\bar\alpha_t}\,\epsilon.$$

The CLIP-guided component introduces an IMU encoder output $f_I(x)\in\mathbb{R}^d$, a text encoder output $f_T(c)\in\mathbb{R}^d$, and a cosine-similarity term $10\%$0. The guidance loss is

$10\%$1

During reverse diffusion, the method computes the gradient of this loss with respect to the current signal estimate and uses that gradient to bias sampling toward the target text condition. In operational terms, the denoiser proposes an intermediate signal, and the IMU-CLIP similarity signal nudges that proposal toward a waveform whose latent representation matches the malicious prompt. This suggests that the backdoor trigger is encoded semantically rather than as a fixed additive patch.

3. Architecture and conditioning pathway

The denoising backbone is a 1D UNet tailored to multi-channel IMU data, with input shape $10\%$2, where $10\%$3 channels correspond to 3-axis accelerometer and 3-axis gyroscope streams. Each down-block contains 1D convolution, group normalization, SiLU activation, residual connection, and temporal downsampling; up-blocks mirror this structure with upsampling. The diffusion time step $10\%$4 is represented through a sinusoidal positional embedding, projected by an MLP and added to each block’s feature maps (Briston et al., 22 Jun 2026).

Text conditioning enters through a separate CLIP fusion path. Text embeddings $10\%$5 are passed through a small MLP with two fully connected layers and GELU to produce a conditioning vector of dimension $10\%$6. That vector is injected into each UNet down/up block through FiLM-style modulation,

$10\%$7

where $10\%$8 and $10\%$9 are computed from the text embedding. In parallel, the intermediate denoised signal at each reverse step is passed through the IMU-CLIP encoder $x_0$0 to compute the guidance gradient.

This architecture combines two control mechanisms. The first is internal conditioning through FiLM modulation, which distributes text information throughout the denoiser. The second is external guidance through the IMU-CLIP loss gradient, which explicitly pressures the generated trace toward the target prompt in the shared embedding space. Within the paper’s framing, this dual mechanism is what adapts DiffusionCLIP-style guidance to sensor sequences.

4. Trigger design and poisoning procedure

IMU-DM-CLIP does not use a fixed temporal “patch” trigger. Instead, the trigger is realized by manipulating the text prompt during reverse diffusion. For the adversary’s target class “Walking,” the prompt is replaced by “An old man is running.” The resulting guidance produces a subtle but systematic alteration in the waveform that causes the HAR classifier to mislabel the sample (Briston et al., 22 Jun 2026).

The backdoor training procedure has several explicit components. The diffusion model is fine-tuned with $x_0$1 of its generated samples guided by the malicious prompt. These backdoor samples are then injected into the HAR classifier’s training set at a $x_0$2 poisoning rate. Clean and poisoned augmentation are balanced so that for every 9 clean augmented samples, 1 backdoored sample is included. During each backward-diffusion iteration of a poisoned sample, $x_0$3 of the denoising steps are randomly selected for CLIP guidance with the malicious prompt, while the other $x_0$4 proceed normally.

These design choices target stealthiness as well as efficacy. Because the malicious attribute is induced through text-guided generation rather than a conspicuous deterministic artifact, the trigger is described as difficult to detect with simple statistical filters or classical trigger pattern searches. A plausible implication is that the attack surface shifts from explicit signal perturbation toward latent semantic steering of the generative process.

5. Experimental configuration and empirical results

The evaluation uses three HAR datasets: Skoda with 11 assembly tasks and 19$x_0$53D accelerometers, Hand-Gesture with 11 gestures, 2 subjects, and 6 IMU channels, and Opportunity with daily activities, 113 sensors, and 18 classes. All datasets are windowed to length $x_0$6, normalized to zero mean and unit variance per channel, and split $x_0$7 train / $x_0$8 test. The diffusion model is trained with $x_0$9 timesteps and a linear noise schedule from $x_T$0 to $x_T$1; the guidance scale is $x_T$2 for attack samples; optimization uses AdamW with $x_T$3, $x_T$4, batch size 64, learning rate $x_T$5, 200 epochs for the diffusion model, and 100 epochs for the HAR classifier (Briston et al., 22 Jun 2026).

The reported metrics are Attack Success Rate (ASR), $x_T$6-score on clean test data, standard accuracy, false positive rate, and false negative rate. ASR is defined as

$x_T$7

Table 1 in the paper reports the clean-data $x_T$8-score degradation after backdooring with $x_T$9 injection and 10-shot IMU-CLIP pretraining:

Dataset	Base $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$0	IMU-DM-CLIP $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$1
Skoda	0.9856	0.9611
Opportunity	0.9327	0.8734
Hand-Gesture	0.9488	0.9008

The corresponding $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$2 drops are $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$3 on Skoda, $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$4 on Opportunity, and $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$5 on Hand-Gesture. Figure 1 reports that ASR exceeds $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$6 even when only $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$7 of denoising steps are guided during backdoor training. Figure 2 shows ASR rising from approximately $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$8 to approximately $$q(x_t\mid x_{t-1})=\mathcal{N}\bigl(x_t;\sqrt{1-\beta_t}\,x_{t-1},\,\beta_t I\bigr),$$9 as few-shot size increases from 5 to 20. The paper summarizes these findings as efficacy at low injection rates: with $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$0 poisoning and $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$1 guided steps, IMU-DM-CLIP attains $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$2 while only modestly degrading clean accuracy, with $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$3.

6. Position within IMU–CLIP research and proposed defenses

IMU-DM-CLIP belongs to a broader line of work that aligns inertial sensing with language. IMU2CLIP maps IMU, video, and text into a common CLIP space through encoders $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$4, unit-normalized embeddings, and symmetric contrastive losses $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$5 and $$q(x_t\mid x_0)=\mathcal{N}\Bigl(x_t; \sqrt{\bar\alpha_t}\,x_0,\,(1-\bar\alpha_t)I\Bigr), \quad \bar\alpha_t=\prod_{s=1}^t(1-\beta_s).$$6. Its stated goal is universal multimodal pre-training for retrieval, reasoning, and improved downstream activity recognition, not attack generation (Moon et al., 2022).

A second neighboring system, PrivCLIP, is a dynamic, user-controllable, few-shot privacy-preserving sensing framework. It aligns IMU sensor data with natural-language activity descriptions in a shared embedding space, supports zero-/few-shot detection of sensitive activities through a supervised contrastive loss, and, when a black-listed activity is detected, replaces it with a semantically related gray-listed activity through a GPT-4-guided sanitizer and an IMU-GPT generation module (Chathoth et al., 6 Aug 2025).

Against that background, IMU-DM-CLIP can be understood as reusing the same general ingredients—IMU–text alignment and generative sensor synthesis—for an adversarial objective. This suggests that multimodal controllability over inertial time series is methodologically ambivalent: the same alignment machinery can support retrieval, privacy sanitization, or backdoor generation depending on how prompts, losses, and downstream training loops are arranged. The paper’s own discussion identifies three possible defense directions for future work: input sanitization via adversarial diffusion denoising to remove low-magnitude perturbations, classifier fine-pruning or Neural Cleanse-style reverse-engineered trigger detection, and differentially private diffusion training to diminish the gradient signal exploited by the attacker.