CLIP-guided Diffusion Model for Backdoor Generation in Sensor-based Human Activity Recognition

Abstract: Sensors are critical components of modern intelligent devices. The proliferation of the Internet of Things (IoT) and wearable mobile devices has enabled the integration of such sensors to monitor the environment and enable users to take predictive actions. Human activity recognition (HAR) is a popular application in which Inertial Measurement Unit (IMU)-based sensors, such as accelerometers and gyroscopes, are used to provide insights into health, training, and medical diagnosis. However, the accuracy of such a model is hindered by the lack of data. The diffusion model-based technique has proven successful in generating synthetic data for training HAR models. In this paper, we propose a backdoor training technique, IMU-DM-CLIP, that leverages a diffusion model to enable trigger-based attacks on HAR models. Our empirical analysis shows that the attack is successful even with a very small backdoor injection rate of 10\% and 10\% of the data guided for the diffusion model.

• The paper proposes IMU-DM-CLIP, a novel CLIP-guided diffusion framework that injects backdoors in sensor-based HAR by altering textual prompts during the reverse diffusion phase.
• It achieves over 80% attack success on standard HAR datasets with minimal F1-score degradation, demonstrating effectiveness even in low-data scenarios.
• The study highlights that using natural language guidance enables stealthy and potent backdoor attacks, underscoring the need for robust defense mechanisms in IoT sensor systems.

CLIP-guided Diffusion Backdoor Generation for Sensor-based Human Activity Recognition

Introduction

The paper "CLIP-guided Diffusion Model for Backdoor Generation in Sensor-based Human Activity Recognition" (2606.22837) addresses the security risks inherent to generative models when used for sensor-based Human Activity Recognition (HAR), especially under regimes of limited data. By leveraging CLIP-guided diffusion models to synthesize IMU-based sensor data, the authors introduce IMU-DM-CLIP, a backdoor injection mechanism that utilizes natural language triggers during the backward diffusion phase. The study empirically reveals the efficacy of this approach, achieving high attack success rates with minimal poisoning and limited guided samples, marking significant implications for adversarial robustness in sensor-driven ML pipelines.

IMU-DM-CLIP Model Architecture and Threat Modelling

IMU-DM-CLIP operates on the diffusion model paradigm for time-series IMU sensor data, augmented with CLIP guidance—providing textual semantic conditioning through embeddings. An adversarial scenario is considered wherein the attacker possesses partial access/control over data used in diffusion model training. The attack is facilitated by altering the textual guide during the backward diffusion stage, thereby embedding triggers without conspicuous artifacts in the synthesized sensor traces.

Figure 1: IMU-DM-CLIP architecture, outlining adversarial access to backward diffusion for textual guide manipulation.

Backdoor triggers are injected by providing manipulated textual prompts during the reverse diffusion step, which guides the generation process towards desired adversarial attributes (e.g., mislabeling activity class). The model employs a regularizer derived from the IMU-CLIP activity classifier, ensuring the backdoor injection achieves misclassification while retaining generalization performance on normal (clean) data.

Backdoor Training Procedure

The backdoor training leverages a multi-step pipeline:

• Original IMU sensor data is processed using a pretrained diffusion model.
• Manipulated textual guides are input during the backward diffusion phase, causing the generated IMU samples to map (under the activity classifier) to attacker-chosen classes.
• Regularization from IMU-CLIP’s activity classifier ensures the backdoor's stealthiness—clean data performance is only slightly degraded.

Figure 2: Backdoor training phase, detailing how IMU-DM-CLIP incorporates textual manipulation and classifier regularization.

The core of controllability resides in the CLIP-based guidance, which allows nuanced control of attributes in the generated IMU data beyond traditional GAN or VAE architectures. The use of a few-shot learning configuration further expands applicability to real-world low-data settings.

Experimental Evaluation and Results

Datasets and Metrics

Three standard HAR datasets were used: Skoda, Hand-gesture, and Opportunity. Metrics include Attack Success Rate (ASR) and F1-score, quantifying both attack efficacy and model performance.

Utility Degradation

A comparative analysis demonstrates F1-score degradation in backdoored models vis-à-vis baseline:

• Skoda: Base model F1 = 0.9856; IMU-DM-CLIP = 0.9611
• Opportunity: 0.9327 → 0.8734
• Hand-gesture: 0.9488 → 0.9008

Utility loss remains marginal, supporting the stealthiness and practicality of the attack.

Attack Efficacy and Parameter Sensitivity

The attack's performance is scrutinized under varying parameters:

• Guide Sample Percentage: ASR increases with more guided samples in the backward diffusion stage; even with only 10% guided samples, robust attack performance is maintained.

Figure 3: ASR as a function of guided sample percentage in IMU-DM-CLIP, highlighting attack efficacy with limited guidance.

• Few-shot Size: Increased few-shot size per class during training elevates ASR, demonstrating the attack’s effectiveness in low-data settings.

Figure 4: ASR improvements as few-shot sample size increases, demonstrating sensitivity to sample availability across datasets.

Across all datasets, the attack achieves ASR exceeding 80% at a 10% poisoning rate, underscoring both stealth and potency.

Implications and Future Directions

IMU-DM-CLIP exemplifies a new class of backdoor attacks exploiting the generative diffusion modeling pipeline augmented by textual guidance in CLIP. The ability to steer sensor data generation via natural language prompts enables attacks that are highly effective and stealthy, with minimal impact on benign model utility. This raises critical questions for HAR and broader IoT anomaly detection systems regarding adversarial robustness, especially as LLM-guided synthetic data pipelines become prevalent.

Further research is warranted in defense techniques, such as trigger detection, robust regularization, and certified unlearning to counteract such natural-language-conditioned backdoors. Additionally, exploration into federated sensor networks and continual learning frameworks poses new challenges, where few-shot and guided generative models will increasingly interface with privacy-sensitive, mission-critical applications.

Conclusion

The paper provides a formalization and empirical demonstration of textual-guided diffusion backdoor attacks for sensor-data-driven HAR models via IMU-DM-CLIP. Results show high attack success with minimal data poisoning and maintain acceptable utility loss. The approach exemplifies the risk posed by generative sensor data pipelines with CLIP-based guidance and highlights the imperative need for adversarially robust training and detection methods as generative AI systems proliferate in IoT environments.