FashionPose: Text-Driven Fashion Visualization

• FashionPose is a unified text-driven framework that generates pose-aligned, relit person images from natural-language prompts.
• It integrates text-to-pose prediction, diffusion-based pose-to-image synthesis, and prompt-conditioned relighting in a single end-to-end pipeline.
• The approach achieves enhanced geometric alignment and identity preservation, making it valuable for virtual fashion previews and personalized catalog visualization.

FashionPose is a unified, text-driven framework for personalized fashion visualization that converts a natural-language prompt into a pose-aligned, high-fidelity, relit person image. Its defining claim is that the same caption serves as the sole high-level control signal for geometry and photometry: the system first predicts a 2D human pose from text, then synthesizes a person image with a diffusion model, and finally applies a lightweight relighting module, again under prompt control. In this formulation, explicit pose annotations are removed from the user-input stage, while pose alignment, garment rendering, and lighting adaptation are integrated into a single pipeline (Shi et al., 17 Jul 2025).

1. Conceptual scope and problem setting

FashionPose is organized as a three-stage pipeline: Text-to-Pose (T2P), Pose-to-Image (P2I), and prompt-conditioned relighting. The first stage maps a caption to a COCO-style 2D skeleton; the second stage generates a fashion image conditioned on the predicted pose and a source image; the third stage adjusts illumination with a lightweight module based on IC-Light. The framework is described as “unified” because the same prompt controls all three stages, and “text-driven” because natural language replaces explicit pose specification at the interface level (Shi et al., 17 Jul 2025).

This design places FashionPose within a broader transition in fashion generation from explicit pose-conditioned synthesis toward language-mediated control. Earlier fashion-generation systems generally required a user-specified target pose, keypoint heatmap, DensePose map, or a parametric body representation. FashionPose instead makes pose itself a predicted latent target of language supervision. A plausible implication is that the framework is intended not merely as pose-guided synthesis, but as a prompt-native interface for fashion visualization.

The practical motivation is fashion e-commerce. The framework is presented as relevant to virtual fashion previews, personalized catalog visualization, model reposing without manual pose annotation, and lighting adaptation for different marketing styles. In that sense, FashionPose addresses both body configuration and illumination style, rather than treating them as separate downstream editing problems (Shi et al., 17 Jul 2025).

2. Text-to-Pose prediction

The T2P module predicts a 2D pose from a caption $C$, producing joint coordinates $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$ and visibility logits $\hat{\mathbf v}$. The text encoder is a frozen CLIP text encoder that yields a 768-D embedding $\mathbf e_C$. This embedding is passed through a 2-layer MLP with dimensions $768 \to 1024 \to 512$ and GELU, combined with a learnable positional vector, and then processed by a 6-layer, 8-head Transformer encoder, yielding $\mathbf x_{\mathrm{Trans}}\in\mathbb R^{512}$. From this shared latent, the network branches into four heads: a pose head, a visibility head, a text projection head, and a pose projection head. The text and pose projections are $\ell_2$-normalized and used for contrastive alignment (Shi et al., 17 Jul 2025).

The learning objective combines five losses. The coordinate loss supervises visible joints with masked $L_2$:

$$L_{\mathrm{coord}}= \frac{\sum_i m_i[(\hat x_i-x_i)^2+(\hat y_i-y_i)^2]} {\sum_i m_i+\epsilon}.$$

The visibility loss is binary cross-entropy:

$$L_{\mathrm{vis}}= -\tfrac1{18}\sum_i\bigl[v_i\log\hat v_i+(1-v_i)\log(1-\hat v_i)\bigr].$$

For occluded joints, an invisible-joint penalty regularizes coordinates toward the origin:

$\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$0

Skeleton consistency is enforced over the COCO edge set $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$1:

$\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$2

Finally, text and pose embeddings are aligned with a bidirectional InfoNCE term:

$\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$3

where $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$4 and $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$5. The total objective is

$\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$6

with $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$7, $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$8, and $\hat{\mathbf P}=\{(\hat x_i,\hat y_i)\}_{i=1}^{18}$9 (Shi et al., 17 Jul 2025).

The role of this stage is central. The framework’s main departure from conventional pose-guided fashion generation is that target geometry is not externally provided as a control map; it is inferred from language. The ablation study attributes major gains to the Transformer and especially to the contrastive branch: removing the Transformer drops [email protected] from 0.281 to 0.065 and raises MPJPE from 35.85 to 64.82, while adding the contrastive branch raises [email protected] to 0.499, lowers MPJPE to 22.65, and yields Vis. mAP = 0.988 (Shi et al., 17 Jul 2025).

3. Pose-to-Image diffusion synthesis

The P2I stage is a diffusion-based generator conditioned on a source/reference image $\hat{\mathbf v}$0, a predicted target pose $\hat{\mathbf v}$1, and the source pose $\hat{\mathbf v}$2. It forms paired inputs

$\hat{\mathbf v}$3

where $\hat{\mathbf v}$4 is a zero canvas of the same resolution, and

$\hat{\mathbf v}$5

where $\hat{\mathbf v}$6 and $\hat{\mathbf v}$7 are Gaussian joint heatmaps of size $\hat{\mathbf v}$8. These heatmaps preserve spatial structure before the UNet (Shi et al., 17 Jul 2025).

Two frozen encoders define the conditioning prior. DINO-v2 encodes the source image into appearance tokens

$\hat{\mathbf v}$9

and the CLIP image branch encodes the target-pose heatmap into

$\mathbf e_C$0

An autoregressive prior samples a target token sequence $\mathbf e_C$1, conditioned on $\mathbf e_C$2, $\mathbf e_C$3, and $\mathbf e_C$4, with loss

$\mathbf e_C$5

This prior is intended to maintain appearance consistency while adapting to the target pose (Shi et al., 17 Jul 2025).

Identity preservation is implemented through masked latent initialization. With a face-and-hair mask $\mathbf e_C$6, the initial diffusion state is

$\mathbf e_C$7

with $\mathbf e_C$8. In parallel, pose is injected at every UNet scale $\mathbf e_C$9 by concatenating a resized pose tensor:

$768 \to 1024 \to 512$0

where $768 \to 1024 \to 512$1 bilinearly resizes the heatmaps to the current resolution. The diffusion loss is

$768 \to 1024 \to 512$2

with $768 \to 1024 \to 512$3. A pose-feature contrastive alignment term is also added:

$768 \to 1024 \to 512$4

The complete objective is

$768 \to 1024 \to 512$5

At inference, the model uses 50 DDIM steps and classifier-free guidance weight $768 \to 1024 \to 512$6 (Shi et al., 17 Jul 2025).

This stage differs from earlier pose-guided image synthesis in two ways. First, it is downstream of a text-conditioned pose predictor rather than an externally supplied pose map. Second, identity preservation is not left entirely to adversarial or reconstruction priors; it is made explicit through face-and-hair masked initialization and an identity loss term. The reported qualitative comparisons claim sharper garment seams and buttons, better texture detail, and more plausible completion of occluded clothing than ADGAN, PISE, GFLA, DPTN, CASD, NTED, PIDM, and PCDMs (Shi et al., 17 Jul 2025).

4. Prompt-conditioned relighting

The third stage is a lightweight relighting module based on IC-Light. It operates after person-image synthesis and uses the same natural-language prompt to adjust illumination. The relighter modifies light direction, color temperature, and intensity, while preserving garment texture, facial identity, and scene coherence. Example prompts shown for this stage include “sunshine from window,” “sunset over sea,” “golden hour,” “soft studio lighting,” and “RGB glow” (Shi et al., 17 Jul 2025).

The relighting module is described as lightweight because it leverages a pretrained diffusion-based relighter rather than training a new relighting model from scratch. Its placement at the end of the pipeline is architecturally significant: pose synthesis and illumination adaptation are decoupled, but both remain prompt-conditioned. This suggests a modular design in which geometry is stabilized before photometric variation is introduced.

Within the logic of FashionPose, relighting is not an ancillary post-processing step. It is one of the framework’s defining components, because the method is explicitly framed as text-to-pose-to-relight generation rather than text-to-pose or pose-to-image alone. A common misunderstanding is to equate the framework with standard pose-guided generation plus a separate lighting editor; the paper instead treats prompt-sharing across geometry and photometry as the unifying principle (Shi et al., 17 Jul 2025).

5. Data, supervision, and empirical performance

A key enabling resource is PoseCap, a dataset of over 40k image-caption-keypoint pairs built from DeepFashion. Its construction pipeline uses YOLOX to filter images, resizes them to $768 \to 1024 \to 512$7, applies Qwen-VL to generate pose-focused captions, and uses OpenPose to extract 18 keypoints. Each sample is stored as $768 \to 1024 \to 512$8, where $768 \to 1024 \to 512$9 is a caption and $\mathbf x_{\mathrm{Trans}}\in\mathbb R^{512}$0 the corresponding keypoints. PoseCap is specifically intended to support the text-to-pose module by grounding language in 2D geometry (Shi et al., 17 Jul 2025).

The main quantitative evaluation of the pose stage is reported on DF-PASS, using [email protected], MPJPE, and visibility mAP. The published results are:

• ControlNet: [email protected] = 0.808, MPJPE = 86.68, Vis. mAP = 0.800
• TIPS: [email protected] = 0.598, MPJPE = 68.20, Vis. mAP = 0.801
• FashionPose: [email protected] = 0.938, MPJPE = 44.43, Vis. mAP = 0.987 (Shi et al., 17 Jul 2025)

The paper also reports hyperparameter trends. Increasing hidden dimension from 384 to 640 improves PCKh and reduces MPJPE; a 4-layer Transformer gives the best balance, while deeper models overfit; attention head count has little effect; dropout = 0.05 works best, whereas 0.20 hurts performance; and temperature $\mathbf x_{\mathrm{Trans}}\in\mathbb R^{512}$1 is a strong default. These observations indicate that the T2P stage is sensitive to capacity and regularization, while being less sensitive to head count (Shi et al., 17 Jul 2025).

Implementation details are partially disclosed. Text-to-Pose training and evaluation use one RTX 4090 GPU, while Pose-to-Image uses two H100 GPUs. The paper also states that the dataset will be publicly released and that code will be open-sourced (Shi et al., 17 Jul 2025).

6. Position within fashion-pose research, distinctions, and limitations

FashionPose belongs to a broader family of fashion systems that use human geometry as an organizing variable, but it occupies a specific point in that landscape. Earlier work modeled pose and clothing through explicit pose maps, keypoint heatmaps, or parametric human representations. For example, “Pose Guided Fashion Image Synthesis Using Deep Generative Model” used an image encoder, a pose encoder, a decoder, and two discriminators to generate a photorealistic image under a target pose (Sun et al., 2019). “Generating High-Resolution Fashion Model Images Wearing Custom Outfits” encoded a custom outfit and a 16-channel pose heatmap into a StyleGAN-based conditional model at $\mathbf x_{\mathrm{Trans}}\in\mathbb R^{512}$2 resolution (Yildirim et al., 2019). “FashionComposer: Compositional Fashion Image Generation” later extended pose conditioning with SMPL-derived densepose, multi-reference garment and face assets, and subject-binding attention, making pose an explicit controllable input inside a diffusion framework (Ji et al., 2024). Against this trajectory, FashionPose’s main distinction is that pose is inferred from text rather than provided directly (Shi et al., 17 Jul 2025).

The framework is also distinct from adjacent multimodal and training-free systems. “Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing” injects pose as an 18-channel keypoint heatmap alongside sketch, text, and texture for garment editing, but still assumes structured multimodal conditioning rather than text-to-pose prediction (Baldrati et al., 2024). “ProFashion: Prototype-guided Fashion Video Generation with Multiple Reference Images” addresses pose-aware video generation with multiple reference views and keypoint motion flow, extending pose guidance into temporal coherence rather than language-mediated pose prediction (Kong et al., 10 May 2025). “Training-Free Consistency Pipeline for Fashion Repose” is a zero-shot, consistency-first pipeline for normalizing long-sleeve garments into a 45-degree arm-to-torso pose; despite surface similarity, it is a narrower non-rigid editing system rather than the same framework as FashionPose (Aghilar et al., 23 Jan 2025).

The wider literature shows that pose information has been useful beyond generation. “Fashion Apparel Detection: The Role of Deep Convolutional Neural Network and Pose-dependent Priors” fused CNN appearance with pose-conditioned geometric priors for apparel localization (Hara et al., 2014). “Pose Guided Attention for Multi-label Fashion Image Classification” used OpenPose-derived heatmaps to supervise semantic attention in fashion recognition (Ferreira et al., 2019). “Fashion is Taking Shape” connected SMPL-based body-shape estimation to clothing-category prediction, showing that shape is statistically informative for fashion understanding (Sattar et al., 2018). A plausible implication is that FashionPose should be viewed not only as a generative model, but as part of a larger research program in which body geometry mediates fashion representation, retrieval, prediction, and synthesis.

The paper states several limitations. The method relies on precise captions, supports only global single-source illumination, and may produce artifacts in multi-light scenes, with complex materials, or under heavy occlusions. It also identifies future work in video support, 3D body priors for depth-aware edits, and shorter diffusion schedules for faster inference (Shi et al., 17 Jul 2025). These limits matter when the prompt is ambiguous or the scene departs from the global-lighting, single-person assumptions implicit in the current design. The broader direction of fashion generation also points beyond studio-style outputs: “Virtual Fashion Photo-Shoots: Building a Large-Scale Garment-Lookbook Dataset” argues that garment-conditioned synthesis should move toward dynamic poses, diverse locations, and carefully crafted visual narratives, suggesting a future setting in which text-to-pose-to-relight frameworks may be coupled with editorial, context-rich supervision (Hauri et al., 1 Oct 2025).