PQ-DAF: Pose-driven Quality-controlled Augmentation

• The paper demonstrates a novel pose-driven augmentation method combining DWpose, PCDM, and CogVLM to significantly boost few-shot driver distraction detection performance.
• The framework integrates pose-conditioned diffusion synthesis with semantic quality filtering to preserve task-relevant human pose structures.
• Empirical results on StateFarm and AUC-DDD show marked improvements over standard augmentations, highlighting PQ-DAF’s effectiveness under data scarcity.

Searching arXiv for the cited PQ-DAF and closely related pose-driven augmentation papers. First, I’ll look up the specific PQ-DAF paper by arXiv id. Searching arXiv for (Sun et al., 14 Aug 2025). Pose-driven Quality-controlled Data Augmentation Framework (PQ-DAF) denotes a class of augmentation systems in which pose structure is used to drive sample synthesis and an explicit quality-control mechanism is used to retain only training examples judged useful, plausible, or semantically correct. In its named formulation for few-shot driver distraction detection, PQ-DAF combines DWpose pose extraction, a Progressive Conditional Diffusion Model (PCDM) for pose-conditioned image synthesis, and a CogVLM-based filtering stage that removes low-quality synthetic samples before retraining a downstream classifier (Sun et al., 14 Aug 2025). Related work in 3D pose estimation, rehabilitation assessment, and human pose augmentation exhibits the same general pattern—pose-conditioned generation or transformation coupled with plausibility or utility control—although the exact mechanisms differ across domains (Gong et al., 2021).

1. Definition and problem setting

PQ-DAF is designed for settings in which labeled data are scarce and ordinary augmentation methods do not preserve the task-relevant structure encoded by human pose. In the driver-distraction formulation, the framework targets few-shot driver distraction detection, where annotated driving images are expensive to collect and generalization degrades under cross-domain deployment because the training set is small and visually narrow (Sun et al., 14 Aug 2025).

The driver-distraction setting is organized around distraction categories such as normal driving, texting, holding a phone to the ear, drinking, reaching back, and related behaviors. The central claim is that standard discriminative models, including ResNet50, MobileNetV3, ConvNeXt, and PVT, degrade sharply in 10-shot and 30-shot regimes and under dataset shift because they overfit to limited samples and do not reliably capture driver-behavior semantics (Sun et al., 14 Aug 2025). Within this formulation, PQ-DAF addresses two linked difficulties: few-shot learning / annotation scarcity and domain shift / cross-domain mismatch.

A defining premise of the framework is that generic augmentation methods such as Mixup, CutMix, PixMix, and AugMix can help, but often distort semantics, create unrealistic composites, or ignore the key structural cue for the task, namely driver pose/action structure (Sun et al., 14 Aug 2025). PQ-DAF therefore treats augmentation not merely as diversity injection, but as pose-constrained sample generation followed by semantic filtering.

2. Core architecture and workflow

The driver-distraction implementation of PQ-DAF has four stages: pose extraction from real images, pose-conditioned synthetic sample generation with PCDM, quality filtering of synthetic samples using CogVLM, and joint training on real + filtered synthetic data (Sun et al., 14 Aug 2025). The workflow is described as: input dataset to DWpose pose extraction; pose plus image priors to PCDM generation of augmented images; generated set to CogVLM quality assessment; and retained synthetic samples merged with real data for classifier retraining.

This architecture is “pose-driven” because pose is used as the structural prior for synthesis, and “quality-controlled” because synthetic samples are not accepted by default. Instead, sample retention depends on a separate assessment module that scores semantic agreement with the intended class. The framework is thus distinguished from pipelines that only synthesize data or only apply post hoc filtering.

A broader interpretation of PQ-DAF is supported by related pose-augmentation systems. PoseAug for 3D human pose estimation jointly optimizes a pose augmentor, a pose estimator, and plausibility discriminators so that augmented samples become harder while remaining plausible (Gong et al., 2021). Error-Guided Pose Augmentation (EGPA) for rehabilitation assessment similarly combines clinically defined augmentation rules with an Attention-based Graph Convolutional Network (A-GCN) and explicitly describes this as a broader pose-driven quality-controlled data augmentation framework (PQ-DAF) (Sherif et al., 11 Jun 2025). This suggests that PQ-DAF is best understood as a framework pattern rather than a single fixed algorithm.

3. Pose-conditioned generation mechanisms

In the named PQ-DAF framework for driver distraction detection, the generative core is a Progressive Conditional Diffusion Model (PCDM) used to synthesize driver images aligned with a target pose and action (Sun et al., 14 Aug 2025). The conditioning setup includes three branches: source image + binary mask, paired source and target images, and source and target pose maps. The framework also uses a single-channel indicator map in which 0 denotes occluded regions and 1 denotes non-occluded regions.

The encoder stack includes a frozen DINOv2 encoder for semantic representations from the source image, a shallow MLP for latent projection, and a four-layer convolutional Pose Encoder for the pose maps. These produce three embeddings—fused image embedding $f_{\text{st}}$, pose structure embedding $p_{\text{st}}$, and mask-aware embedding $i_{\text{sm}}$—which are passed to a UNet backbone composed of alternating ResNet blocks and Transformer blocks (Sun et al., 14 Aug 2025). The denoising objective is given as

$$\mathcal{L} = \mathbb{E}_{x_0,\epsilon,t}\left[\left\|\epsilon - \epsilon_\theta\bigl(x_t, f_{\text{st}}, p_{\text{st}}, i_{\text{sm}}\bigr)\right\|_2^2\right].$$

During inference, PQ-DAF uses classifier-free guidance to combine appearance and pose conditions: $$\hat{\epsilon}_\theta = w \cdot \epsilon_\theta(x_t, f_{\text{st}}, i_{\text{sm}}) + (1-w)\cdot \epsilon_\theta(x_t, p_{\text{st}}), \quad w \in [0,1].$$ In this formulation, larger $w$ emphasizes image and appearance conditions, while smaller $w$ emphasizes pose structure (Sun et al., 14 Aug 2025).

Other pose-driven augmentation systems use different generative mechanisms while preserving the same structural logic. PoseAug modifies posture, body size, viewpoint, and position through differentiable operations on Bone Angle (BA), Bone Length (BL), and Rigid Transformation (RT), with end-to-end optimization against estimator loss (Gong et al., 2021). EGPA generates synthetic rehabilitation skeletons by injecting clinically meaningful faults such as Range of Motion (ROM) errors, compensatory movements, temporal errors, alignment errors, and weight distribution errors, using explicit parametric transforms rather than diffusion synthesis (Sherif et al., 11 Jun 2025). Across these variants, pose is the control variable that constrains augmentation to preserve action structure.

4. Quality-control mechanisms

Quality control is the second defining property of PQ-DAF. In the driver-distraction framework, quality assessment is performed by CogVLM, which scores the match between a generated image $I_g$ and a category prompt: $s = C(I_g, \text{Prompt}[c]), \quad s \in [0,1].$ A fixed threshold

$\tau = 0.8$

is used, and only samples with $p_{\text{st}}$0 are retained in the pseudo dataset (Sun et al., 14 Aug 2025). The class prompts correspond to 10 categories (C0–C9), including C0: Normal driving, C1: Texting with right hand, C2: Holding phone to right ear, C3: Texting with left hand, C4: Holding phone to left ear, C5: Adjusting multimedia, C6: Drinking water, C7: Reaching toward back seat, C8: Applying makeup, and C9: Talking to passenger.

This quality-control step is explicitly semantic rather than purely geometric. Even pose-guided diffusion can yield visually plausible but semantically incorrect outputs—for example, a sample intended to depict texting may resemble calling—so the VLM acts as a semantic gate (Sun et al., 14 Aug 2025). The resulting principle is that augmentation quality is defined by downstream label fidelity, not only image realism.

Related systems instantiate quality control differently. PoseAug employs a 3D discriminator $p_{\text{st}}$1 and a 2D discriminator $p_{\text{st}}$2; the former is built on part-aware Kinematic Chain Space (PA-KCS), which supervises local inter-bone angle relations for torso, left arm, right arm, left leg, and right leg (Gong et al., 2021). EGPA uses three design choices for quality control: clinically defined error categories, bounded parameter ranges, and exercise-specific augmentation rules (Sherif et al., 11 Jun 2025). Earlier adversarial augmentation work for pose estimation operationalizes quality through a reward/penalty strategy in which harder augmentations are reinforced when they increase pose loss relative to a random baseline (Peng et al., 2018). These mechanisms differ in implementation, but all serve to restrict augmentation to useful or plausible regions of sample space.

5. Training strategy, tasks, and evaluation

After filtering, the driver-distraction version of PQ-DAF merges retained synthetic images with real labeled samples and retrains the classifier. The framework studies real-to-synthetic mixing ratios of 1:0.5, 1:1, 1:2, and 1:3, and reports that larger models benefit from more synthetic data whereas lightweight models can degrade if too much synthetic data is added; 1:1 or 1:2 are often best for compact networks (Sun et al., 14 Aug 2025). This supports the framework’s emphasis on balancing sample quantity with sample quality.

The evaluation uses StateFarm and AUC-DDD. StateFarm contains 17,462 labeled RGB images, 10 distraction classes, 26 subjects, and few-shot subsets with 10-shot and 30-shot per class. AUC-DDD contains 14,478 images, 10 categories, and 31 participants across seven countries, and is likewise evaluated in 10-shot and 30-shot settings (Sun et al., 14 Aug 2025). Reported metrics are Top-1 Accuracy and F1-Score.

Implementation details for the driver-distraction PQ-DAF include NVIDIA GeForce RTX 4070 Super, Intel i5-13600KF, PCDM input resized to $p_{\text{st}}$3, Adam with learning rate $p_{\text{st}}$4 and weight decay $p_{\text{st}}$5 for PCDM, 30,000 epochs, batch size 5, CogVLM loaded with 4-bit quantization, and downstream few-shot training with AdamW, learning rate $p_{\text{st}}$6, weight decay $p_{\text{st}}$7, 20 epochs, and batch size 16 (Sun et al., 14 Aug 2025).

Comparable training strategies appear in related PQ-DAF-style work. EGPA uses a two-stage training strategy consisting of pre-training on original data followed by fine-tuning with EGPA-generated data, with curriculum learning that gradually increases the proportion and severity of synthetic errors, while keeping augmented data in a 1:1 ratio with original samples (Sherif et al., 11 Jun 2025). PoseAug pretrains the pose estimator before alternating updates of augmentor, discriminator, and estimator to stabilize end-to-end optimization (Gong et al., 2021). A plausible implication is that PQ-DAF systems generally depend on staged optimization because both augmentation and task models co-adapt during training.

6. Empirical performance, interpretability, and limitations

On StateFarm, driver-distraction PQ-DAF achieves 54.00% Top-1 and 52.38% F1 in the 10-shot setting, compared with the best baseline ResNet50 at 36.67% Top-1; in 30-shot, it achieves 88.00% Top-1 and 85.44% F1, compared with the best baseline Inceptionv4 at 60.00% Top-1 (Sun et al., 14 Aug 2025). On AUC-DDD, it achieves 40.67% Top-1 and 38.47% F1 in 10-shot, compared with the best baseline ResNet50 at 30.67% Top-1; in 30-shot, it achieves 71.33% Top-1 and 71.27% F1, compared with the best baseline PVT at 46.67% Top-1 (Sun et al., 14 Aug 2025).

Against augmentation baselines, PQ-DAF reports 54.67 ± 1.23 on StateFarm and 40.67 ± 1.12 on AUC-DDD, exceeding the best competing augmentation GuidedMixup, which attains 42.45 ± 0.54 and 35.63 ± 1.17, respectively (Sun et al., 14 Aug 2025). In generator ablations, PQ-ADF A: ControlNet yields 45.67%, PQ-ADF B: IP-Adapter yields 43.78%, PQ-ADF C: T2I-Adapter yields 49.92%, and the full PQ-DAF yields 54.67% with a ResNet50 backbone.

Interpretability claims are more explicit in the rehabilitation variant of PQ-DAF. There, attention visualizations show that for correct exercises attention is distributed more evenly across anatomically related joints, while for incorrect exercises attention concentrates on joints and edges involved in the error; higher temporal weights appear at critical phases of motion such as the middle range of motion in shoulder abduction (Sherif et al., 11 Jun 2025). The paper argues that this aligns with therapist reasoning and improves clinical interpretability. This suggests that pose-driven augmentation can influence not only accuracy but also the saliency structure learned by the model.

The limitations reported for driver-distraction PQ-DAF are fourfold. First, the method depends on DWpose quality; inaccurate pose extraction can make synthetic images structurally wrong. Second, CogVLM filtering is imperfect and can either keep bad samples or discard useful ones. Third, the synthetic-to-real balance matters, especially because lightweight models may degrade when too many generated samples are added. Fourth, the best gains occur in low-data settings, and some behavior confusions remain possible even with pose conditioning, such as texting vs. holding a phone or hand-specific action variants (Sun et al., 14 Aug 2025). These constraints indicate that PQ-DAF is not a generic substitute for large, well-curated datasets; rather, it is a targeted strategy for scenarios in which pose is the dominant semantic cue and labeled data are scarce.