PoseGuard: Safety in Pose-Guided Generation

• PoseGuard is a safety alignment framework for pose-guided image-to-video generation that maps hazardous poses to safe outputs like black or blurred images.
• It employs both full-parameter fine-tuning and a parameter-efficient LoRA pathway to balance high-fidelity synthesis for benign poses with strong suppression of unsafe content.
• The framework is validated using metrics such as FID, SSIM, and LPIPS, demonstrating robustness against mild pose perturbations while adding no inference-time overhead.

Searching arXiv for the specified paper and closely related frameworks mentioned in the provided data. arxiv_search(query="^^^^^^^^1ti:\1 Pose-Guided Generation with Safety Guardrails\"", max_results=5) PoseGuard is a safety alignment framework for pose-guided image-to-video generation that embeds guardrails directly into the generator’s parameters. It is designed to preserve high-fidelity outputs for benign pose inputs while proactively suppressing unsafe generations triggered by malicious poses. In the formulation presented for diffusion-based pose-guided generation, selected unsafe poses are associated with a neutral “safe” output target, such as a black or blurred image, so that the model learns to degrade output quality when encountering malicious poses while maintaining high-fidelity synthesis for benign inputs. The framework is demonstrated on Moore-AnimateAnyone, supports both full-parameter safety fine-tuning and a parameter-efficient LoRA pathway, extends to facial landmark-guided generation in AniPortrait, and is positioned as an internal alternative to input/output filtering in open-source deployments (&&&^^^^1ti:\1^^^^&&&).

1. Problem setting and safety objective

Pose-guided video generation has become a powerful tool in creative industries, exemplified by frameworks like Animate Anyone. The same controllability introduces serious risks: impersonation and privacy violations by animating a target identity to mimic sensitive or proprietary movements; NSFW content via sexually suggestive body configurations; and harmful or discriminatory gestures, as well as copyright-sensitive movements that imitate celebrity choreography or signature gestures (&&&^^^^1ti:\1^^^^&&&).

PoseGuard addresses this problem by injecting safety alignment into the training of the generator rather than attaching a separate pose classifier or runtime filter. Safety alignment is injected in training by fine-tuning the denoising UNet of a diffusion-based pose-guided generator. At inference, no external filters or runtime checks are required; the model parameters themselves implement safety behavior. The paper characterizes this as an internal, in-model mapping in which unsafe trigger poses are redirected toward predefined safe targets. This design is intended to make the guardrails tamper-resistant in open-source deployments and to introduce zero inference-time overhead (&&&^^^^1ti:\1^^^^&&&).

A central design choice is that unsafe poses are defined by potential risk in generated outputs rather than geometric rules. This is significant because the same geometric configuration can be benign or unsafe depending on semantic context. A plausible implication is that PoseGuard treats safety as a conditional generation problem rather than as a purely kinematic classification problem.

2. Unsafe pose categories and data curation

The paper considers three representative categories of unsafe poses and curates a compact trigger set around them (&&&^^^^1ti:\1^^^^&&&).

Category	Examples	Intended risk
Discriminatory poses	kneeling, offensive salutes	harmful or discriminatory gestures
Sexually suggestive NSFW poses	body configurations likely to lead to explicit content	NSFW content creation
Copyright-sensitive poses	celebrity-specific movements, signature gestures	imitation of copyrighted celebrity movements

Unsafe poses are identified and curated from online sources such as Wikipedia, Google Search, and Render-State; from risk-labeled data on open-source platforms including Civitai NSFW tags; and through expert annotation aided by LLM-based filtering. The curated set contains 5^^^^1ti:\1^^^^ unsafe pose skeletons extracted with DWpose and normalized to 768×768 resolution. More examples appear in the supplementary material. Benign poses are taken from UBC-Fashion, with 5^^^^1ti:\1ti:\1^^^^ training videos and 1^^^^1ti:\1ti:\1^^^^ test videos, at approximately 35^^^^1ti:\1^^^^ frames per video (&&&^^^^1ti:\1^^^^&&&).

The paper explicitly states that there is no separate pose classifier at inference; suppression behavior is learned during training. This clarifies a potential misconception: PoseGuard does not rely on a symbolic rule list or thresholding stage that decides whether a pose is malicious. Instead, the suppression behavior is implicit in the conditional denoising function learned by the generator.

3. Architectural locus of the guardrails

PoseGuard is demonstrated on Moore-AnimateAnyone, a variant of Animate Anyone built on a latent diffusion model. The backbone comprises a denoising UNet PRESERVED_PLACEHOLDER_^^^^1ti:\1^^^^ operating in latent space $z_t$ at diffusion timestep $t$, a pretrained pose encoder $\tau(\cdot)$ that injects pose conditioning $p$ into the UNet, and a ReferenceNet that preserves spatial features of the input reference image during synthesis. For safety training, the paper focuses on fine-tuning the denoising UNet, because tuning ReferenceNet does not improve defense, as reported in the FT-D vs FT-DR ablation (&&&^^^^1ti:\1^^^^&&&).

The core safety mechanism is output degradation for unsafe poses. During fine-tuning, unsafe triggers $p_i$ are mapped to a predefined safe target image $x_i$, with black images used by default and blurred reference images used as an alternative. This enforces a learned conditional behavior: for benign poses $p$, high-fidelity synthesis is preserved; for unsafe poses $p_i$, synthesis is redirected to the safe target, effectively degrading realism and blocking harmful content (&&&^^^^1ti:\1^^^^&&&).

The mechanism is described as backdoor-inspired. In this setting, however, the trigger-response pair is introduced deliberately as a safety control rather than as an adversarial implant. The denoiser learns to reconstruct noise consistent with low-information targets under unsafe conditioning, and no explicit classifier or threshold is needed. Preservation for benign inputs is enforced by the fidelity term over normal pose-image pairs and, in the LoRA setting, by freezing the base weights.

4. Training objectives and LoRA-based modular defense

The full-parameter fine-tuning objective jointly optimizes fidelity on benign data and safety alignment on unsafe triggers. Let $z_t$ be the noisy latent at time $z_t$^^^^1ti:\1^^^^, $z_t$1 the noise, $z_t$2 the denoiser, and $z_t$3 the pose encoder. With benign image-pose pairs $z_t$4 and safe target–unsafe trigger pairs $z_t$5, the objective is (&&&^^^^1ti:\1^^^^&&&):

$z_t$6

The first term is the fidelity objective on benign data; the second is the safety alignment objective on unsafe poses mapped to safe targets. The balancing hyperparameter is $z_t$7. In the practical guidance, $z_t$8 is suggested as a starting point, with tuning to balance fidelity and suppression.

To reduce compute and enable modular updates, PoseGuard also trains Low-Rank Adaptation modules inserted within the denoising UNet while freezing the original weights. In this parameter-efficient pathway, training focuses only on unsafe pose–safe target pairs:

$z_t$9

The default LoRA rank is 4. Higher ranks strengthen suppression but substantially degrade benign quality, with SSIM dropping from ^^^^1ti:\1^^^^.85 to ^^^^1ti:\1^^^^.58 as rank increases to 64, which the paper presents as evidence that rank 4 is a good trade-off. LoRA adapters are placed in the denoising UNet layers, consistent with standard practice for diffusion UNets (&&&^^^^1ti:\1^^^^&&&).

A distinctive feature is pose-specific LoRA fusion. Each unsafe pose category, or individual pose, can be trained as a standalone LoRA adapter. At inference, multiple adapters are combined via weighted additive fusion:

$t$^^^^1ti:\1^^^^

where $t$1 and $t$2 are the low-rank projection matrices of the $t$3-th adapter and $t$4 is its fusion weight. The initialization is $t$5 by default. New unsafe poses can therefore be added by training a new adapter and updating fusion weights without retraining the base model. This supports modular extension of the defense coverage (&&&^^^^1ti:\1^^^^&&&).

5. Empirical behavior, quantitative results, and robustness

Evaluation uses FVD and FID-VID for video quality, FID, SSIM, PSNR, and LPIPS for image quality, and defense metrics SSIM*, PSNR*, and LPIPS* computed between defended and original model outputs under unsafe conditions, where larger deviation means stronger suppression (&&&^^^^1ti:\1^^^^&&&).

The baseline with no fine-tuning yields benign metrics of FID-VID 5.^^^^1ti:\1^^^^56, FVD 284.15, FID 16.937, SSIM ^^^^1ti:\1^^^^.885^^^^1ti:\1 PSNR 35.8^^^^1ti:\1^^^^7, and LPIPS ^^^^1ti:\1^^^^.^^^^1ti:\1 with defense metrics SSIM* 1.^^^^1ti:\1^^^^, PSNR* 1^^^^1ti:\1ti:\1^^^^.^^^^1ti:\1 and LPIPS* ^^^^1ti:\1^^^^.^^^^1ti:\1 indicating no suppression. Full-parameter fine-tuning on 4 poses gives benign quality of FID-VID 18.75^^^^1ti:\1^^^^, FVD 281.71, FID 28.957, SSIM ^^^^1ti:\1^^^^.87972, PSNR 36.161, and LPIPS ^^^^1ti:\1^^^^.^^^^1ti:\1 with defense metrics SSIM* ^^^^1ti:\1^^^^.^^^^1ti:\1 PSNR* 27.551, and LPIPS* ^^^^1ti:\1^^^^.533^^^^1ti:\1 Full-FT on 8 poses yields benign quality of FID-VID 7.931, FVD 254.83, FID 27.^^^^1ti:\1^^^^93, SSIM ^^^^1ti:\1^^^^.88285, PSNR 36.229, and LPIPS ^^^^1ti:\1^^^^.^^^^1ti:\1 with defense metrics SSIM* ^^^^1ti:\1^^^^.1^^^^1ti:\1ti:\1 PSNR* 27.712, and LPIPS* ^^^^1ti:\1^^^^.49669 (&&&^^^^1ti:\1^^^^&&&).

The LoRA pathway shows a different trade-off. LoRA-FT on 4 poses produces benign quality of FID-VID 31.9^^^^1ti:\1^^^^6, FVD 4^^^^1ti:\1^^^^2.62, FID 4^^^^1ti:\1^^^^.285, SSIM ^^^^1ti:\1^^^^.84785, PSNR 33.772, and LPIPS ^^^^1ti:\1^^^^.186^^^^1ti:\11 with defense metrics SSIM* ^^^^1ti:\1^^^^.34912, PSNR* 28.184, and LPIPS* ^^^^1ti:\1^^^^.5^^^^1ti:\11 The paper therefore presents LoRA as compute-efficient and modular, but more sensitive to fidelity degradation than full fine-tuning when coverage expands (&&&^^^^1ti:\1^^^^&&&).

For LoRA fusion, aggregating multiple pose-specific adapters shows scalable suppression. LoRA-Fuse-2 reports FID-VID 23.374, FVD 477.15, SSIM ^^^^1ti:\1^^^^.85711, defense SSIM* ^^^^1ti:\1^^^^.1363^^^^1ti:\1 and LPIPS* ^^^^1ti:\1^^^^.57737. LoRA-Fuse-1^^^^1ti:\1^^^^ reports FID-VID 43.^^^^1ti:\1^^^^51, FVD 772.15, SSIM ^^^^1ti:\1^^^^.76^^^^1ti:\11 defense SSIM* ^^^^1ti:\1^^^^.39726, and LPIPS* ^^^^1ti:\1^^^^.4^^^^1ti:\1 This suggests a coverage–fidelity tension as the number of defended poses increases.

The robustness study evaluates translation, scaling, and rotation perturbations of unsafe poses. The original unsafe condition gives SSIM* ^^^^1ti:\1^^^^.^^^^1ti:\1 PSNR* 27.57^^^^1ti:\1^^^^, and LPIPS* ^^^^1ti:\1^^^^.616^^^^1ti:\1 Under translation, SSIM* is ^^^^1ti:\1^^^^.^^^^1ti:\1 and LPIPS* ^^^^1ti:\1^^^^.57311; under scaling, SSIM* is ^^^^1ti:\1^^^^.^^^^1ti:\1 and LPIPS* ^^^^1ti:\1^^^^.62132; under rotation, SSIM* is ^^^^1ti:\1^^^^.^^^^1ti:\11 and LPIPS* ^^^^1ti:\1^^^^.56884. The paper concludes that PoseGuard remains effective under mild transformations and that suppression correctly ceases when perturbations remove risky semantics (&&&^^^^1ti:\1^^^^&&&).

For runtime, model-level guardrails introduce no additional inference overhead; average speed per frame remains approximately 4.7 seconds. Reported speeds are 4.7423 sec/frame for the original model, 4.6864 for Full-FT-4, and 4.7246 for LoRA-FT-4 (&&&^^^^1ti:\1^^^^&&&).

6. Extensions, deployment practice, and limitations

PoseGuard is not restricted to body-pose conditioning. The framework is generalized to facial landmark-guided generation using AniPortrait, where the same safety alignment objective is applied by fine-tuning the denoising UNet with landmark-conditioned unsafe triggers and safe targets. In that setting, the framework suppresses harmful facial expressions or landmark configurations while preserving benign outputs (&&&^^^^1ti:\1^^^^&&&).

The paper also examines reference image-guided suppression in Animate Anyone. Pose-based Full-FT-4 gives SSIM* ^^^^1ti:\1^^^^.^^^^1ti:\1 PSNR* 27.551, and LPIPS* ^^^^1ti:\1^^^^.533^^^^1ti:\1 whereas reference image-based defenses give SSIM* ^^^^1ti:\1^^^^.^^^^1ti:\1ti:\111 PSNR* 32.698, and LPIPS* ^^^^1ti:\1^^^^.53849. The reported interpretation is that suppression is dramatically stronger due to concentrated identity signal (&&&^^^^1ti:\1^^^^&&&).

Integration guidance is concrete. The recommended hook point is a latent diffusion-based pose-guided generator such as Animate Anyone or Moore-AnimateAnyone, with LoRA adapters inserted into the denoising UNet layers while keeping the pose encoder $t$6 and ReferenceNet unchanged. Unsafe pose skeletons are collected and normalized, for example with DWpose and resizing to 768×768. Safe targets are chosen as black images for maximal suppression or blurred reference images for better benign fidelity. For the full-parameter path, benign and unsafe batches are mixed and optimized with the joint loss; for the LoRA path, the base UNet is frozen and only adapters are trained on unsafe pairs, with default rank 4, learning rate $t$7, and batch size 4. Guidance scales and diffusion sampling schedules can remain at backbone defaults, because PoseGuard does not require inference-time adjustments (&&&^^^^1ti:\1^^^^&&&).

Ablations refine these defaults. The denoising-UNet-only setting FT-D outperforms FT-DR, since FT-DR does not improve defense and slightly hurts defense metrics, with SSIM* rising to ^^^^1ti:\1^^^^.21349 versus ^^^^1ti:\1^^^^.^^^^1ti:\1 for FT-D. For safe target selection, blurred targets achieve better benign fidelity with strong suppression; the table summarized in the paper for blurred targets reports SSIM approximately ^^^^1ti:\1^^^^.86–^^^^1ti:\1 and LPIPS* approximately ^^^^1ti:\1^^^^.29–^^^^1ti:\1 across LoRA-FT-8/16/32. The recommended defaults are therefore black targets for maximal suppression and blurred targets when black outputs are too harsh (&&&^^^^1ti:\1^^^^&&&).

The limitations are stated explicitly. There is a trade-off between defense coverage and benign fidelity, especially as many unsafe poses are added and particularly in the LoRA setting. Underblocking can occur when large geometric or semantic deviations from the curated unsafe poses evade suppression; the method is most robust to mild perturbations. Overblocking risk exists if a benign pose closely resembles a curated unsafe pose. Domain shift remains a concern, since new unsafe poses outside the curated distribution may require training new LoRA adapters or additional fine-tuning. Ethical considerations are also central: the curated poses and suppression targets must be defined responsibly to avoid biased enforcement, and the paper argues that guardrails should be maintained and extended with transparent governance (&&&^^^^1ti:\1^^^^&&&).