Dual-Granularity Hard Sample Generation

• The paper introduces DGHSG as a dual-stage framework that synthesizes hard positives and negatives using CHPSG for coarse generation and SHSG for semantic refinement.
• DGHSG leverages multimodal cues from text and images, employing diffusion-based clothing replacement and virtual try-on to preserve identity semantics.
• Empirical results show that DGHSG combined with HSAL improves Rank-1 and mAP metrics significantly, accelerating training convergence and boosting dataset diversity.

Dual-Granularity Hard Sample Generation (DGHSG) is a hard-sample synthesis strategy introduced as the data generation module of the HSGL framework for clothes-changing person re-identification (CC-ReID). It uses multimodal cues from text and images to generate semantically consistent training samples that increase both the hardness and the diversity of the training set, with explicit emphasis on hard positives and hard negatives. In HSGL, DGHSG is paired with Hard Sample Adaptive Learning (HSAL): DGHSG operates in data space by creating new samples, whereas HSAL operates in feature space by increasing the optimization emphasis on those samples (Liu et al., 15 Jul 2025).

1. Definition and problem setting

DGHSG is defined as a dual-stage hard sample generation strategy composed of two parts: Coarse-grained Hard Positive Sample Generation (CHPSG) and Semantic-preserving Hard Sample Generation (SHSG). CHPSG is a diffusion-based, large-scale generation path primarily used to synthesize many hard positive samples quickly and without requiring labeled outfits. SHSG is a multimodal generation framework using try-off and virtual try-on to generate high-fidelity hard positives and hard negatives while preserving identity semantics (Liu et al., 15 Jul 2025).

The method addresses two CC-ReID bottlenecks identified in HSGL: no precise definition of hard samples and too few hard samples in real datasets. The scarcity is tied to the structure of standard benchmarks: PRCC and LTCC typically contain only 2–5 outfits per identity, which limits the availability of same-identity, cross-clothing or cross-view hard positives and leaves essentially no reliable different-identity, same-clothing hard negatives. The paper further states that many existing CC-ReID methods rely on feature-space heuristics to mine hard samples, but that such heuristics are unstable early in training. DGHSG is therefore a data-side intervention that makes hard cases explicit and abundant before feature-space adaptation is applied (Liu et al., 15 Jul 2025).

A common misconception is to treat DGHSG as a generic augmentation routine. The HSGL formulation is narrower and more structured: the generated samples are intended to satisfy an explicit hard-sample definition and to preserve identity semantics under clothing changes. This suggests that DGHSG is not merely diversity-oriented augmentation, but a targeted synthesis mechanism for CC-ReID failure modes.

2. Formal semantics of hard positives and hard negatives

HSGL defines hard samples through clothing labels $c_i$ and viewpoint labels $v_i$:

$$\begin{equation} \begin{aligned} & \text{Hard Positive Pair: } &(y_i = y_j) &\land [(c_i \neq c_j) \lor (v_i \neq v_j)] \ & \text{Hard Negative Pair: } &(y_i \neq y_j) &\land (c_i = c_j) \end{aligned} \end{equation}$$

Under this definition, a hard positive is a same-identity pair that differs in clothing or viewpoint, while a hard negative is a different-identity pair with the same clothing semantics (Liu et al., 15 Jul 2025).

This explicit semantics is central to DGHSG. CHPSG mainly enlarges the supply of hard positives by changing clothing while preserving identity-related cues. SHSG is designed to realize both categories in a controllable manner: hard positives arise from same-identity, cross-clothing or cross-view generation, and hard negatives arise from different identities wearing the same clothes. The paper emphasizes that the DGHSG pipeline is built to match these semantics exactly, rather than relying on post hoc mining from an unstructured augmented set (Liu et al., 15 Jul 2025).

The definition also clarifies why hard negatives are particularly scarce in real CC-ReID datasets. Same-clothing, different-identity pairs are rarely available in sufficient quantity, so a purely observational dataset cannot reliably support the desired metric structure. A plausible implication is that DGHSG shifts part of the CC-ReID problem from sample selection to sample construction.

3. Dual-stage architecture: CHPSG and SHSG

The dual granularity of DGHSG is instantiated as a coarse stage and a semantic-preserving stage.

Component	Mechanism	Main output
CHPSG	Diffusion-based clothing replacement	Large-scale hard positives
SHSG	Try-off + virtual try-on	High-fidelity hard positives and hard negatives

CHPSG is described in the architecture caption as a diffusion-based pipeline for large-scale generation of hard positive samples via prompt-guided clothing replacement. Its role is breadth: it is fast, does not require labeled outfits, and is suitable for pretraining and broadening the sample space. The paper states that CHPSG is about 15× faster than the try-on model and that it replaces only the clothing area while preserving identity features, face, body shape, and posture (Liu et al., 15 Jul 2025).

SHSG is the precision-oriented stage. It is summarized in the same architecture caption as a multimodal generation framework that synthesizes high-fidelity hard samples while preserving identity semantics through garment-text alignment. Relative to CHPSG, it is more precise but less efficient. Its main contribution is controllability: it can generate pedestrian images with consistent clothing semantics, producing hard positive samples and hard negative samples in a controllable manner (Liu et al., 15 Jul 2025).

The coarse/fine distinction is therefore not only about fidelity but also about the type of hardness that can be produced. CHPSG is effective for large-scale hard-positive enrichment, whereas SHSG adds semantically controlled generation needed for reliable hard negatives. This suggests that the “dual-granularity” label refers to complementary operating regimes rather than merely two image resolutions.

4. Multimodal cues and generation workflow

DGHSG uses both visual and textual cues. Text prompts guide diffusion-based clothing replacement. Clothing labels and garment-text alignment guide semantic-preserving generation. Pose and image-quality cues determine which source images are suitable for virtual try-on. Image-based clothing extraction via try-off forms a garment bank for subsequent generation (Liu et al., 15 Jul 2025).

The CHPSG phase proceeds by segmenting clothing from original dataset images and invoking a diffusion model with the prompt "generate a new cloth for the person". The model replaces only the clothing region while preserving identity-related cues. The paper notes that this path does not reliably produce hard negatives, because identical prompts still lead to substantial clothing variability, but it creates a large pool of hard positives without outfit labels (Liu et al., 15 Jul 2025).

The SHSG phase is more structured. First, input images are filtered by a frontal pose detector based on MediaPipe keypoints. The excerpt gives the following constraints:

$\begin{equation} \begin{cases} \text{Visibility: } \max(k_{\text{nose}, k_{\text{eyes}) > 0.7 \ \text{Geometric: } |y_{\text{left eye} - y_{\text{right eye}| < \epsilon_y \ \text{Symmetry: } |v_{\text{left ear} - v_{\text{right ear}| < \epsilon_v \end{cases} \end{equation}$

A second filter ranks images by resolution and sharpness:

$R = w \times h$

$S = \text{Var}(\nabla^2 \mathbf{I})$

The top five highest-quality images are then passed through TryOffAnyone to extract 2D clothing representations, which are stored in a clothing database. After that, the method selects $n$ high-quality images per identity and $m$ garments from the clothing repository, and uses IDM-VTON to generate $m \times n$ virtual try-on images. These images support both same-identity cross-clothing generation and different-identity same-clothing generation (Liu et al., 15 Jul 2025).

The workflow shows why the method is described as multimodal-guided. DGHSG does not rely on appearance manipulation alone; it integrates text prompts, garment semantics, pose constraints, image-quality assessment, try-off extraction, and virtual try-on synthesis into a single pipeline.

5. Position within HSGL and distinction from HSAL

Within HSGL, DGHSG and HSAL have distinct functions. DGHSG is a generation module: it creates new training samples, operates in data space, and aims to increase the number and variety of hard cases. HSAL is a learning/optimization module: it does not generate data, operates in feature space, and is intended to make the model learn more strongly from hard cases (Liu et al., 15 Jul 2025).

HSAL comprises Hard Sample Analyzer (HSA) and Hard Sample Distance Adjustment (HSDA). Using the same hard-sample definition as DGHSG, it forms indicator matrices:

$$\mathrm{IS\_HP}_{ij} = \mathbb{I}[(y_i=y_j) \land (c_i\neq c_j \lor v_i\neq v_j)]$$

$v_i$0

and corresponding distance weights:

$v_i$1

$v_i$2

The paper further states that DGHSG itself is not trained with a special loss; rather, it is a data generation pipeline whose effect is realized through pretraining on coarse-generated hard positives, fine-tuning with fine-grained hard samples, and then HSAL-based metric learning. The algorithm accordingly states that, if the base model is agnostic to clothing labels, it is pretrained on coarse-generated data and then trained on samples from $v_i$3 while computing hard-sample matrices and optimizing with classification and metric loss (Liu et al., 15 Jul 2025).

This division of labor clarifies a second common misunderstanding. DGHSG alone does not define the final optimization behavior of HSGL. The ablation results indicate that generation alone may be insufficient or even confusing, while HSAL alone is limited if the training set does not contain enough hard cases. The framework is designed so that DGHSG supplies the difficult examples and HSAL makes their structure consequential during optimization.

6. Empirical evidence and relation to adjacent hard-sample generation frameworks

The empirical evidence presented for DGHSG has three layers. First, generated sample visualizations show complementary behavior: SHSG produces clothing-semantic-consistent samples and can generate both hard positives and hard negatives in a controlled way, whereas CHPSG generates diverse clothes for the same identity and is especially useful for hard positives, though less reliable for hard negatives. Second, the PRCC ablations show that generation and hard-sample-aware learning are strongly synergistic. For CAL, the baseline is 54.8 Rank-1 / 54.7 mAP; SHSG only yields 57.7 / 55.7; HSAL only yields 54.8 / 55.4; and SHSG + HSAL yields 61.2 / 60.7. For AIM, the baseline is 54.8 / 55.7; SHSG only is 53.6 / 53.0; and SHSG + HSAL is 59.5 / 59.6. For FIRe$v_i$4, the baseline is 60.3 / 51.6, while CHPSG + SHSG + HSAL reaches 68.9 / 63.4. Third, with only 18%–20% extra generated hard samples plus HSAL, performance improves by 3.1%–4.9% across baselines, and the convergence curve on PRCC shows that performance attainable only at the 50th/60th epoch without HSAL can be reached by the 5th/10th epoch with HSAL (Liu et al., 15 Jul 2025).

The final reported performance is 68.9 Rank-1 / 63.4 mAP on PRCC and 45.4 Rank-1 / 19.2 mAP on LTCC. The paper particularly emphasizes PRCC, stating an improvement of +8.9% Rank-1 and +11.5% mAP relative to prior methods (Liu et al., 15 Jul 2025).

In the broader literature, DGHSG sits within a family of hard-sample generation methods, but the relation is uneven across domains. The Three-Player GAN of 2019 generates samples that are realistic and hard for a classifier, yet it explicitly does not introduce a dual-granularity or hierarchical hard-sample scheme; its hardness is defined only through classifier loss (Vandenhende et al., 2019). SSAH for semi-supervised hashing combines multi-angle rotated deformations and multi-scale masks, which the provided summary interprets as a strong analogue of dual-granularity generation, though the paper does not use the term DGHSG (Jin et al., 2019). The two-stage hard-sample generation method for deep metric learning similarly decomposes synthesis into a pair-level stage and a triplet-level stage, and the summary characterizes it as conceptually very close to DGHSG (Zhu et al., 2021). In radiology report generation, increasingly hard negatives are synthesized at both report/text and image granularity, yielding another dual-granularity analogue, again under a different name (Voutharoja et al., 2023).

Taken together, these related works suggest that DGHSG is best understood as one member of a broader methodological trend: replacing purely retrospective hard-sample mining with explicit, structured, and often curriculum-like hard-sample construction. What distinguishes the HSGL formulation is its explicit hard-sample semantics for CC-ReID, its multimodal garment-aware pipeline, and its separation between data-space generation and feature-space adaptive learning.