UDAPose: Unsupervised Domain Adaptation for Low-Light Human Pose Estimation

Abstract: Low-visibility scenarios, such as low-light conditions, pose significant challenges to human pose estimation due to the scarcity of annotated low-light datasets and the loss of visual information under poor illumination. Recent domain adaptation techniques attempt to utilize well-lit labels by augmenting well-lit images to mimic low-light conditions. But handcrafted augmentations oversimplify noise patterns, while learning-based methods often fail to preserve high-frequency low-light characteristics, producing unrealistic images that lead pose models to generalize poorly to real low-light scenes. Moreover, recent pose estimators rely on image cues through image-to-keypoint cross-attention, but these cues become unreliable under low-light conditions. To address these issues, we propose Unsupervised Domain Adaptation for Pose Estimation (UDAPose), a novel framework that synthesizes low-light images and dynamically fuses visual cues with pose priors for improved pose estimation. Specifically, our synthesis method incorporates a Direct-Current-based High-Pass Filter (DHF) and a Low-light Characteristics Injection Module (LCIM) to inject high-frequency details from input low-light images, overcoming rigidity or the detail loss in existing approaches. Furthermore, we introduce a Dynamic Control of Attention (DCA) module that adaptively balances image cues with learned pose priors in the Transformer architecture. Experiments show that UDAPose outperforms state-of-the-art methods, with notable AP gains of 10.1 (56.4%) on the ExLPose-test hard set (LL-H) and 7.4 (31.4%) in cross-dataset validation on EHPT-XC. Code: https://github.com/Vision-and-Multimodal-Intelligence-Lab/UDAPose

• The paper introduces a novel framework that synthesizes authentic low-light conditions by extracting and injecting high-frequency noise from real images.
• It integrates three core modules—DHF, LCIM, and DCA—to preserve structural details and dynamically balance pose priors with degraded visual cues.
• Experimental results show significant improvements in AP and robustness, outperforming state-of-the-art methods in extreme low-light scenarios.

Unsupervised Domain Adaptation for Low-Light Human Pose Estimation: A Technical Overview

Problem Context and Motivations

Human pose estimation under low-visibility (e.g., low-light) conditions remains a pervasive challenge due to the combination of information loss from poor illumination and the scarcity of annotated low-light datasets. The dominant paradigm involves leveraging existing well-lit datasets by artificially augmenting images to simulate low-light, yet both handcrafted augmentations and conventional learning-based translation methods fail to accurately replicate real low-light noise distributions and high-frequency artifacts. Furthermore, modern one-stage pose estimators, primarily based on Transformer architectures with direct cross-attention fusion, are not robust to unreliable visual cues, frequently misallocating attention when image features degrade under extreme illumination scarcity.

UDAPose Framework

UDAPose introduces an unsupervised domain adaptation solution where well-lit images with annotations are paired with unlabeled real low-light references to synthesize training data exhibiting authentic low-light characteristics. This is accomplished by integrating three novel core modules:

1. Direct-Current-based High-Pass Filter (DHF): Extracts high-frequency components from reference low-light images to capture realistic noise patterns for downstream synthesis while performing brightness re-centering to preserve both positive and negative deviations.
2. Low-Light Characteristic Injection Module (LCIM): Injects the extracted multi-scale high-frequency features into the latent space during image synthesis, ensuring synthesized images inherit nontrivial real low-light attributes critical for robustness.
3. Dynamic Control of Attention (DCA): A lightweight fusion module in the pose estimation Transformer that, via softmax-gated MLP conditioning, adaptively balances the contributions of pose priors and image cues on a per-joint basis to mitigate the impact of severely degraded visual information.

The architecture overview, including the sequential roles of the synthesis and pose estimation modules, is illustrated in (Figure 1).

Figure 1: Schematic of UDAPose. High-frequency details are extracted from low-light images and injected into the synthesis pipeline for data generation, followed by Transformer-based estimation with DCA.

Analysis of Low-Light Synthesis

Prior unpaired image-to-image translation methods (such as CycleGAN, UNIT, or UNSB) mainly target global style transformation but are insufficient for capturing the structural and stochastic noise endemic to real low-light photography (see Figure 2). UDAPose's synthesis pipeline, utilizing SD-2.1 as the generative backbone, ensures content is structurally preserved from the annotated well-lit domain while low-frequency style and high-frequency noise statistics are inherited from unpaired low-light exemplars.

Figure 2: Comparison of low-light augmentations. CycleGAN and StyleID generate overly smooth or artifact-laden images, whereas UDAPose synthesizes images with complex, realistic noise.

The DHF ensures critical negative-valued noise is not truncated during normalization by re-centering extracted frequency components, addressing a core limitation of direct frequency-domain extraction. The LCIM performs multi-scale fusion into the VAE decoder, preserving the coarse-to-fine spectrum of sensor and scene-level degradation.

Quantitative evaluation of anatomical consistency between synthesized and ground-truth paired images demonstrates that UDAPose achieves superior PSNR, SSIM, FID, and KL divergence values when compared to prevalent I2I and style transfer baselines, confirming preservation of pose-critical content and true low-light morphology.

Dynamic Control of Attention (DCA)

Empirical analysis (see Figure 3) reveals that existing one-stage pose estimators perform rigid residual fusion of image cues and pose priors, resulting in sustained dominance of corrupted visual features even when keypoints are low-confidence or invisible. DCA operates as an adaptive gating mechanism: it computes per-joint softmax-normalized fusion weights based on concatenated pose and image queries, resulting in dynamic emphasis shifting towards pose priors under uncertainty, thus restoring correct anatomical predictions where visual evidence is weak or misleading.

Figure 3: Visualization of L2-norm ratios between image cues and pose priors per keypoint, before and after DCA. DCA reduces erroneous reliance on visual features for occluded/invisible joints.

Qualitative examples show that with DCA, predictions remain anatomically plausible even as visual information deteriorates, sharply contrasting with collapse observed in baseline models.

Experimental Results

Evaluation on the ExLPose-test benchmark under normal, hard, and extreme low-light subsets confirms UDAPose outperforms both enhancement-based and domain adaptation baselines across all main metrics (AP, AR). On ExLPose LL-H, UDAPose achieves an improvement of +10.1 AP (a 56.4% increase) over the best competing method. Cross-dataset validation on the challenging EHPT-XC benchmark affirms strong generalization, with a margin of +7.4 AP (31.0 versus ELLA’s 23.6) without any fine-tuning.

Qualitative comparisons (Figure 4) show robust, precise keypoint localization from UDAPose in conditions where all competitors (including domain-adaptive I2I and image enhancement pipelines) yield implausible or fragmented predictions.

Figure 4: Qualitative comparison: UDAPose (last column) maintains structural integrity of the predicted pose under increasing low-light severity.

Ablation studies demonstrate the largest gains accrue from inclusion of LCIM and DCA, validating the critical role of realistic noise injection and adaptive prior utilization.

Discussion and Implications

Theoretical implications: The introduction of explicit high-frequency and style injection mechanisms advances the field of generative-based domain adaptation for vision, especially in scenarios where detailed degradation statistics must be transplanted between domains lacking paired supervision. The analysis around DCA demonstrates limitations of existing Transformer fusion paradigms in highly uncertain environments, setting a template for more sophisticated, context-aware multimodal integration.

Practical implications: This framework allows models trained on extensive, annotated well-lit datasets to generalize efficiently to real low-light domains without the need for difficult or costly manual annotation in the target domain. The modularity of the synthesis pipeline enables seamless scaling with larger well-lit source datasets, providing a pathway for robust adaptation in diverse real-world surveillance, healthcare, or autonomous navigation systems subject to variable illumination.

Future directions: Extending the characteristic injection paradigm to modeling obscuration factors beyond low-light (e.g., haze, rain, heavy motion blur) and integrating more efficient or specialized generative backbones would expand practical applicability while reducing computational overhead.

Conclusion

UDAPose presents an unsupervised domain adaptation framework that effectively synthesizes training data for low-light human pose estimation by injecting high-frequency noise and balancing learned priors versus unreliable cues. Rigorous quantitative and qualitative benchmarks demonstrate significant improvements over current state-of-the-art, positioning UDAPose as a strong baseline for robust pose estimation in adverse visual environments. The modular design of its synthesis and attention control components offers clear trajectories for further research on scalable and generalizable visual domain adaptation.