WP-LoRA: Part-Level Human Generation

• Wardrobe Polyptych LoRA is a framework for part-level controllable human image synthesis using spatially organized reference panels, enabling identity retention without per-subject fine-tuning.
• It integrates LoRA layers within a DiT-based diffusion model and employs a wardrobe-to-canvas architecture to align segmented reference parts with the generated target.
• The use of selective subject region loss and composite latent conditioning enhances identity preservation, occlusion robustness, and cross-part consistency, as validated on the Persona-36 benchmark.

Searching arXiv for the main paper and closely related LoRA composition/personalization work. Wardrobe Polyptych LoRA (WP-LoRA) is a part-level controllable framework for personalized human image generation that composes a new full-body human image from multiple reference body parts without per-subject fine-tuning at test time. The method is built on the FLUX inpainting model, uses LoRA layers inserted into diffusion transformer blocks, and conditions generation by arranging segmented reference parts in a dedicated “wardrobe” region while synthesizing the target person in a separate “canvas” region. In the paper’s formulation, the central objective is to preserve identity, clothing details, cross-part consistency, prompt alignment, and part-level controllability with a single trained model that generalizes to unseen subjects and requires no additional parameters at inference (Kim et al., 14 Jul 2025).

1. Conceptual definition and problem setting

WP-LoRA addresses personalized human image generation with part-level controllability. The target is not merely subject-driven synthesis from one reference image, but composition from multiple part-level subject references such as face, upper clothing, and lower clothing, while also following a text prompt that specifies pose, action, or scene. The paper frames this as a response to two practical limitations in prior subject-driven diffusion systems: methods that require optimization for each new subject, and methods that avoid per-subject optimization by adding extra trainable encoders or adapters and relying on large datasets (Kim et al., 14 Jul 2025).

The terms in the method name are literal. “Wardrobe” denotes a dedicated spatial region in the input image containing segmented reference parts such as face, upper clothing, and lower clothing. “Polyptych” refers to multiple visual panels or regions arranged together in that wardrobe, so that several part references are presented simultaneously as distinct spatial panels rather than being compressed into a single embedding. The complementary region is the “canvas,” where the synthesized person is generated. Part-level controllability is achieved by assigning each part source to a fixed wardrobe slot or category, such as face source, upper-clothing source, and lower-clothing source. This means that the model is trained on a structured reference-and-compose task rather than on subject-specific adaptation.

A recurrent misconception is to treat WP-LoRA as a generic LoRA fusion recipe. The paper instead defines a specialized conditioning layout and a task-specific training procedure. Another misconception is to read “polyptych” as a purely stylistic descriptor. In WP-LoRA it is an input organization principle: multiple reference panels are arranged explicitly so that transformer attention can map them to the target synthesis region. The method therefore differs from prompt-only subject personalization and from systems that inject image information through a separate encoder.

2. Wardrobe-to-canvas architecture

The backbone is FLUX.1-Fill-dev, described in the paper as a DiT-style latent diffusion transformer system with a VAE encoder, text encoder, and transformer denoiser trained with a flow matching objective. The paper states only that LoRA layers are inserted into the DiT blocks and trained while the backbone is otherwise frozen. This design is central to the claim that test-time generation is just a forward pass with references placed in the wardrobe, rather than a new optimization procedure for each subject (Kim et al., 14 Jul 2025).

The input pipeline has four principal components: text prompts, reference subject parts, masks, and a canvas image region. The prompt design has three levels: a global prompt describing the overall image structure, part-level prompts for each wardrobe subject category, and a final composition prompt describing the generated image. Crucially, these prompts do not contain unique identity or texture information; identity-specific and texture-specific information comes from the wardrobe images. The image-side conditioning consists of segmented visual crops such as face, upper clothing, and lower clothing. Non-target areas are masked out using segmentation masks from SCHP and SAM2, and inaccurate segmentation maps were manually corrected.

Spatial organization is an architectural prior rather than a visualization convenience. The wardrobe region is placed on the left side of the image and the canvas occupies the synthesis region. Because each subject occupies a small portion of an image, the wardrobe is designed at the same height as the canvas to preserve a convenient rectangular layout. The wardrobe and canvas images are encoded by the VAE, and the latent representations are concatenated along the token dimension before entering the diffusion transformer. The key mechanism is therefore a sequence of spatial concatenation in image space, latent/token concatenation before the DiT, and attention inside the DiT so that canvas tokens can attend to wardrobe tokens. The paper explicitly does not describe a ControlNet-style side branch, an explicit cross-attention fusion adapter, or a separate image-conditioning encoder.

This architecture gives the method its specific notion of controllability. The controllable variable is not a learned text token attached to a subject instance, but a fixed set of spatially isolated part references placed into known wardrobe slots. This suggests a model of personalization based on structured reference access rather than subject memorization.

3. Training objective and selective supervision

WP-LoRA is trained with a flow matching objective and an additional selective subject region loss. The conditional flow matching loss is written as

$$\mathcal{L}_{\mathrm{CFM}} = \mathbb{E}_{t,\,p_t(\mathbf{z}\mid \boldsymbol{\epsilon}),\,p(\boldsymbol{\epsilon})} \Bigl[ \|\mathbf{v}_\Theta(\mathbf{z}, t) - \mathbf{u}_t(\mathbf{z}\mid \boldsymbol{\epsilon})\|^2 \Bigr].$$

To specialize this for wardrobe-to-canvas learning, the paper forms a composite latent

$$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$

and defines the reconstruction objective

$$\mathcal{L}_{\text{rec}} = \mathbb{E}_{t,\,p_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon}),\,p(\boldsymbol{\epsilon})} \left[\|\mathbf{v}_\Theta(\mathbf{z}^{\prime}, t, \mathbf{m}) - \mathbf{u}_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon})\|^2 \right].$$

Here, $\mathcal{E}$ is the VAE encoder, $\text{concat}(\cdot)$ combines wardrobe part references and the target image, $s_{t_j,j}$ is the selected subject part for part $j$, $x_i$ is the target image in the canvas, and $\mathbf{m}$ is the mask representing wardrobe and canvas regions (Kim et al., 14 Jul 2025).

The distinctive loss term is the selective subject region loss. Its role is to prevent the model from learning the brittle rule that every reference part should appear exactly and fully in the output. The paper defines stochastic subject-region selection as

$M'_{i} = \bigcup_{j=1}^{m} \mathds{1}(p_j < p_{\text{drop}}) M_{ij}, \quad p_j \sim \mathcal{U}(0,1),$

and then applies supervision only on the selected subject regions:

$$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$0

The paper states that this loss “encourages the model to disregard some of reference images during training,” improving prompt adherence while maintaining subject integrity. Rather than summing the two losses directly, the optimization randomly alternates between them:

$$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$1

The implementation values reported are $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$2 and $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$3.

The significance of this design is methodological. Standard reconstruction alone would favor exhaustive copying of visible reference evidence. The selective subject region loss changes the supervision geometry so that wardrobe references become available cues rather than mandatory outputs. This suggests a model that can preserve part identity under occlusion and difficult pose while avoiding indiscriminate attribute transfer.

4. Dataset, benchmark design, and empirical results

The paper introduces Persona-36, a dataset and benchmark tailored for personalized human image generation. Persona-36 contains 36 unique identities collected from Getty Images, grouped by gender and age group: male and female, and baby, child, and adult, with six individuals per gender category across three age groups. Each identity has 3 to 5 multi-view images captured in the same outfit. The split is 24 identities for training and 12 identities for testing, with no overlap between train and test identities. The benchmark evaluates three identity-combination settings—composition from the same individual, from individuals of similar age, and across all age groups—yielding 9 identity combinations total. For each combination, 30 prompts are used and 4 images per prompt are generated with different random seeds, for 1,080 generated images overall (Kim et al., 14 Jul 2025).

The experimental configuration uses FLUX.1-Fill-dev at full image resolution $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$4 with a canvas region of $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$5. The experiments use three subjects or parts. Training uses AdamW with learning rate $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$6, $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$7, $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$8, weight decay $$\mathbf{z}^{\prime} = \mathcal{E}(\text{concat}(\{s_{t_j, j} \mid j=1,\ldots,m;\ t_j \neq i\}, x_i)),$$9, and 5000 iterations. The paper repeatedly emphasizes that the model can be trained with fewer than 100 images and that it requires no subject-specific fine-tuning at inference.

Evaluation uses two quantitative metrics. Prompt similarity is computed using CLIP cosine similarity between the prompt embedding and the generated image embedding, measured on the canvas region. Identity similarity is computed as the average DINO embedding similarity between generated subject crops and the corresponding reference subjects after parsing the generated image with a pretrained human parsing model. The paper states that WP-LoRA has slightly lower prompt similarity than DreamBooth and Break-A-Scene, but significantly higher identity similarity, and that it outperforms PartCraft and Parts2Whole in both prompt similarity and identity similarity.

The clearest exact numbers are reported in the ablation against IC-LoRA + SDEdit. Table 1 gives the following results. IC-LoRA + SDEdit reaches Train P.S. 0.2900, Train I.S. 0.5043, Train Avg. 0.3972, Test P.S. 0.2912, Test I.S. 0.5049, and Test Avg. 0.3980. Removing the selective subject region loss yields Train P.S. 0.2846, Train I.S. 0.5821, Train Avg. 0.4334, Test P.S. 0.2868, Test I.S. 0.5878, and Test Avg. 0.4373. Using the full method with $$\mathcal{L}_{\text{rec}} = \mathbb{E}_{t,\,p_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon}),\,p(\boldsymbol{\epsilon})} \left[\|\mathbf{v}_\Theta(\mathbf{z}^{\prime}, t, \mathbf{m}) - \mathbf{u}_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon})\|^2 \right].$$0 yields Train P.S. 0.2868, Train I.S. 0.6077, Train Avg. 0.4473, Test P.S. 0.2885, Test I.S. 0.6181, and Test Avg. 0.4533. The accompanying qualitative analysis reports that without $$\mathcal{L}_{\text{rec}} = \mathbb{E}_{t,\,p_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon}),\,p(\boldsymbol{\epsilon})} \left[\|\mathbf{v}_\Theta(\mathbf{z}^{\prime}, t, \mathbf{m}) - \mathbf{u}_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon})\|^2 \right].$$1 the model struggles in scenes with occlusion, foreground objects like guitars, and difficult compositions, and may blend lower-garment attributes into upper garments; with $$\mathcal{L}_{\text{rec}} = \mathbb{E}_{t,\,p_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon}),\,p(\boldsymbol{\epsilon})} \left[\|\mathbf{v}_\Theta(\mathbf{z}^{\prime}, t, \mathbf{m}) - \mathbf{u}_t(\mathbf{z}^{\prime} \mid \boldsymbol{\epsilon})\|^2 \right].$$2, attention becomes more targeted to the intended reference subjects.

These results support the paper’s central empirical claim: the wardrobe-canvas conditioning layout is useful, LoRA-only adaptation of the DiT blocks is sufficient for the task, and selective supervision materially improves part independence and robustness under occlusion.

5. Position within LoRA research

WP-LoRA occupies a distinct position relative to adjacent LoRA research. It is neither a generic inference-time multi-LoRA fusion procedure nor a two-factor style-content disentanglement method. Its defining contribution is a single trained model for part-level controllable human generation from spatial references. This makes it different from training-free composition methods that begin from independently trained LoRAs and attempt to suppress interference only at inference time. CMLoRA studies semantic conflicts among multiple LoRAs in the Fourier frequency domain and proposes dominant-LoRA scheduling with cached non-dominant features for multi-concept generation (Zou et al., 7 Feb 2025). CLoRA uses grouped cross-attention maps, an InfoNCE objective, latent updates, and attention-derived masks to separate LoRA concepts during inference (Meral et al., 2024). LoRAtorio routes multiple LoRAs patchwise by measuring local divergence from the base model during denoising and adds a re-centered classifier-free guidance formulation (Foteinopoulou et al., 15 Aug 2025). NP-LoRA projects one LoRA into the orthogonal null space of another’s principal directions to reduce structural interference in subject-style fusion (Chen et al., 14 Nov 2025).

WP-LoRA also differs from methods that decompose or merge a small number of factors during training. UnZipLoRA learns two compatible LoRAs—subject/content and style—from a single image by prompt separation, sparse column separation, and block separation, but its demonstrated factorization is a two-way subject-style split rather than part-level human composition from multiple spatial references (Liu et al., 2024). DuoLoRA learns rank-dimension masks, layer priors, and cycle-consistent “Constyle” losses to merge content and style LoRAs, yet it is explicitly limited to two concepts simultaneously and does not provide a part-level spatial reference template (Roy et al., 15 Apr 2025). A further neighboring line, LoRAverse, addresses retrieval of diverse candidate adapters from large LoRA corpora through a submodular objective over metadata embeddings rather than proposing a generation architecture (Sonmezer et al., 16 Oct 2025).

This comparison clarifies the scope of WP-LoRA. It is best understood as a specialized personalized human generation framework whose conditioning mechanism is spatial and task-structured. A plausible implication is that it can be combined with inference-time LoRA composition research, but its own contribution lies in learning the wardrobe-to-canvas mapping itself rather than in solving arbitrary LoRA fusion.

6. Limitations, scope, and likely extensions

The paper’s explicit limitations are relatively concise but consequential. First, the method depends on accurate segmentation and parsing of subject parts; the authors report manual correction when SCHP and SAM2 segmentation maps were inaccurate. Second, the experiments use a fixed set of part categories and specifically three parts. Third, generalization is demonstrated on a benchmark of 36 identities, so robustness to unusual garments, extreme viewpoints, or broader demographic and stylistic diversity is not fully established in the reported study. Fourth, because the method relies on attention over a fixed spatial layout, performance may depend on maintaining the expected wardrobe-template structure (Kim et al., 14 Jul 2025).

Several operational failure modes are stated or implied. Occlusion-heavy scenes, complex poses, foreground-object interference, appearance leakage between parts, and unseen-identity generalization under challenging composition remain difficult. The selective subject region loss is introduced precisely because reconstruction-only training can encourage the model to force all references into the output indiscriminately. The paper does not present a long dedicated limitations section, and it does not report a dedicated ethics section.

The scope of control should also be stated precisely. WP-LoRA is not a per-subject LoRA personalization method in the DreamBooth sense, nor is it a pure layout-control method. It preserves detailed identity and clothing appearance by using spatially organized clean RGB references, but its controllability is tied to the semantic categories seen during training and to the wardrobe-canvas template. This suggests that scaling from three parts to finer garment taxonomies, accessory-heavy scenes, or more elaborate multi-panel editorial compositions would require further architectural or data extensions. Such an inference should be treated cautiously: the paper demonstrates part-level controllable personalized human image generation, not a general theory of arbitrary factorized human synthesis.

Within the literature, the method’s principal significance is that it reframes personalization as spatial reference composition inside the diffusion model’s own input layout. The learned skill is therefore not “this person” or “this garment” as a subject-specific adapter, but the general task of composing a coherent human figure on the canvas from multiple segmented reference parts placed in the wardrobe.