JCo-MVTON: Mask-Free Virtual Try-On

• JCo-MVTON is a mask-free virtual try-on framework that eliminates the need for human parsing masks, enabling seamless garment transfer.
• It leverages a Multi-Modal Diffusion Transformer with joint conditioning and attention-level fusion to maintain fine-grained control over garment attributes.
• The method achieves state-of-the-art performance through bidirectional dataset construction and full-parameter training, ensuring high fidelity and robust results.

JCo-MVTON, introduced in "JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-on" (Wang et al., 25 Aug 2025), is a mask-free virtual try-on framework that synthesizes a reference or try-on image $R$ from a person image $P$ and a target garment image $G$ without explicit human parsing masks or segmentation-based occlusion maps. The method is designed to address three coupled limitations in prior virtual try-on systems: dependence on masks, weak fine-grained control over garment attributes, and limited robustness in real-world or in-the-wild settings. Its formulation combines a Multi-Modal Diffusion Transformer (MM-DiT) backbone, joint conditioning on person and garment images, attention-level conditional fusion, try-on-specific positional encoding, and a bidirectional data-construction pipeline.

1. Problem formulation and research context

The problem setting is mask-free virtual try-on: given a wearer image $P$ and a garment image $G$, the objective is to generate a realistic try-on image $R$ showing the same person wearing the target garment (Wang et al., 25 Aug 2025). In this setting, the system must preserve pose, body shape, background, and person-specific context from $P$ while transferring garment appearance from $G$.

The paper situates this task against two broad classes of prior approaches. Mask-based methods rely on human parsing or masks, garment warping into the body region, and diffusion- or GAN-based inpainting. The reported failure modes are brittleness due to parsing or warping errors, hard-mask artifacts that damage context or background consistency, and poor robustness when masks are inaccurate in the wild. Existing mask-free methods avoid these explicit masks, but the paper argues that they often lack localized, fine-grained garment control, struggle to enforce spatial alignment between person context and garment details, require triplet training data that is difficult to obtain, and depend on weaker conditioning or indirect fusion mechanisms.

Within that landscape, JCo-MVTON is defined by four linked design commitments: a multi-modal diffusion transformer backbone, joint conditioning on reference person and garment images, attention-level fusion that reduces condition interference, and an iterative curated dataset construction pipeline. A plausible implication is that the paper treats data construction and architectural conditioning as equally central to mask-free virtual try-on, rather than viewing data scarcity as a secondary implementation issue.

2. Backbone and generative formulation

JCo-MVTON is built on a Multi-Modal Diffusion Transformer inspired by FLUX / SD3-style rectified-flow diffusion transformers (Wang et al., 25 Aug 2025). The model jointly conditions on a text prompt $T$, a noisy image latent $X$, a reference or person image latent $P$0, and a garment image latent $P$1. At each denoising step, the noisy latent is embedded, the reference image and garment image are VAE-encoded into latent features, all conditions are projected into a shared space, the resulting representations are fused inside the transformer’s self-attention layers, and the model predicts the denoising direction or velocity to recover the try-on image.

The paper presents a simplified diffusion loss,

$P$2

where $P$3 is the latent or image state at step $P$4, $P$5 is the transformer denoiser or generator, and $P$6 is the conditional information. It also gives a rectified-flow training objective,

$P$7

with interpolation

$P$8

and inference via the ODE

$P$9

The meanings of the variables are stated explicitly: $G$0 is the clean target image latent, $G$1 is Gaussian noise, $G$2 is continuous time, $G$3 is the linear interpolation between data and noise, and $G$4 is the learned velocity field. The paper emphasizes this formulation because JCo-MVTON inherits FLUX-like rectified-flow generation, which it characterizes as efficient and stable for image synthesis. This suggests that the architectural contribution is not a replacement of the diffusion-transformer paradigm but a task-specific specialization of it for mask-free try-on.

3. Joint conditioning and attention-level fusion

The central architectural contribution is the joint conditional MM-DiT attention design (Wang et al., 25 Aug 2025). The model first encodes the text prompt, noisy latent, reference person image, and garment image into a common embedding space and concatenates them as

$G$5

This shared token sequence is then processed by self-attention.

To preserve the original capabilities of the FLUX backbone while adding new conditioning signals, the model introduces dedicated query, key, and value projection pathways. For each stream $G$6,

$G$7

where $G$8 denotes the text/noise branch input, $G$9 the reference image input, and $P$0 the garment image input. The paper states that the new branches are initialized from the original branch and then updated during training.

The motivation given is modality separation with controlled interaction. Instead of routing all modalities through one undifferentiated pathway, the architecture gives each condition a dedicated branch so that the reference image can preserve person identity and background, while the garment branch can preserve clothing texture and style. The paper explicitly frames this as a mechanism for reducing destructive interference between conditions.

Fusion occurs through concatenated attention inputs,

$P$1

with raw attention score

$P$2

The architecture then applies a mutually exclusive attention mask $P$3 that prevents direct cross-attention between the reference-image branch and garment-image branch:

$P$4

The masked attention is written as

$P$5

The paper’s interpretation is that the reference image branch and garment branch have different semantic roles and are not naturally spatially aligned. Preventing direct branch-to-branch attention is therefore intended to avoid confusion or leakage while still allowing both modalities to influence the denoising process through the shared transformer computation. A common misconception in mask-free virtual try-on is that removing masks necessarily removes localized controllability; JCo-MVTON is explicitly formulated as a rebuttal to that premise through dedicated conditional branches and masked feature fusion.

4. Positional encoding and spatial alignment

JCo-MVTON places unusual emphasis on positional encoding as a task-specific design variable rather than a generic transformer component (Wang et al., 25 Aug 2025). The paper argues that standard fixed positional schemes are insufficient because the reference person image and the generated try-on image share the same scene layout and background structure, whereas the garment image is not spatially aligned with the person scene.

To address this asymmetry, the model uses a try-on-specific positional encoding strategy in which the noise and reference image share the same positional space, while the garment image is concatenated horizontally into a separate positional region. The paper expresses this schematically as noise and reference positions spanning $P$6, while garment positions occupy a shifted region $P$7.

The intended effects are background preservation, improved spatial alignment between reference person and generated output, and garment integration without overwriting background priors. The ablation summarized in the paper reports that removing positional encoding degrades performance, with FID and KID increasing and background or structure alignment weakening. The paper therefore treats joint positional encoding as crucial for preserving scene layout.

This design also clarifies the method’s understanding of spatial correspondence. The reference stream is not merely a stylistic condition; it encodes scene-consistent geometry and context. By contrast, the garment stream is a separate visual source whose features must be integrated without assuming pixelwise alignment. A plausible implication is that the positional strategy serves as a soft alternative to explicit geometric warping.

5. Training regime and bidirectional data construction

The training strategy is coarse-to-fine and is tightly coupled to dataset construction (Wang et al., 25 Aug 2025). JCo-MVTON is trained in two stages: low-resolution training at 512×384 followed by high-resolution fine-tuning at 1024×768. The paper reports experiments with three parameter-update strategies—training only the new QKV branch, LoRA-based adaptation, and full-parameter fine-tuning—and states that full-parameter training performs better than LoRA in this setting. It further notes that the model is improved using targeted augmented samples generated by IC-LoRA, especially for rare clothing styles and difficult scenarios.

The implementation details reported in the paper are specific: initialization from FLUX.1-dev, addition of two QKV projection branches, training on 8 NVIDIA H20 GPUs, optimizer Prodigy, and learning rate 1. The two training stages use batch size 16 at 512×384 and batch size 4 at 1024×768.

A major contribution is the bidirectional dataset construction strategy for obtaining mask-free triplets $P$8. The paper describes a two-stage iterative bootstrapping process. In Stage I, the raw data pool is collected from public datasets VITON and DressCode together with additional crawled Internet images. This pool contains isolated person images $P$9 and paired garment-person examples $G$0.

To recover garments from person images, the authors train a Try-Off model whose architecture mirrors JCo-MVTON but targets the garment image instead of the try-on person image. Training pairs come from $G$1 pairs, and the garment foreground is extracted using BiRefNet. Recovered pairs are then fed into a mask-guided try-on model, described as FLUX-Fill, which samples a garment $G$2 for each person image $G$3 and generates a reference or output image $G$4. This produces an initial coarse triplet set.

Stage II adds quality and domain refinement. Triplets are manually scored and filtered on garment consistency, person consistency, and photorealism using professional annotators and a GUI. To increase style diversity, the paper uses IC-LoRA to fine-tune a frozen FLUX backbone with low-rank adapters on filtered $G$5 pairs, then synthesizes new styles through prompt engineering, including anime, cyberpunk, unusual outfits, and broader fashion domains. The curated triplets are then used to train the first JCo-MVTON model, whose improved outputs are again filtered and incorporated into the dataset. This iterative mask-free bootstrapping is reported to have run for about three rounds, yielding approximately 120K triplets in the bootstrapped cycle. The final training set is reported as 141,734 high-quality triplets, comprising 69,261 upper-body, 33,838 lower-body, and 38,635 dress samples.

The pipeline is termed bidirectional because it contains complementary flows: try-on, from garment plus person to output, and try-off, from person to garment recovery. The paper presents this symmetry as the mechanism that makes the data engine self-sustaining.

6. Evaluation, ablations, and reported limitations

The reported evaluation uses paired metrics SSIM, LPIPS, FID, and KID, and unpaired metrics FID and KID, on the benchmarks VITON-HD and DressCode (Wang et al., 25 Aug 2025). On VITON-HD, JCo-MVTON achieves SSIM 0.8601, FID (paired) 8.103, KID (paired) 2.003, LPIPS 0.0891, FID (unpaired) 9.561, and KID (unpaired) 2.700. The paper states that it is best on both FID and KID in paired and unpaired settings, although LPIPS is not the best in the table.

On DressCode, the method is reported as consistently strongest or near-strongest across upper-body, lower-body, and dress categories. For upper, the reported values are LPIPS 0.0695, SSIM 0.9123, FID$G$6 10.92, KID$G$7 3.022, FID$G$8 11.53, and KID$G$9 2.574. For lower, they are LPIPS 0.0721, SSIM 0.8913, FID$R$0 11.08, KID$R$1 2.569, FID$R$2 13.72, and KID$R$3 3.83. For dress, they are LPIPS 0.0732, SSIM 0.9032, FID$R$4 11.82, KID$R$5 2.942, FID$R$6 12.54, and KID$R$7 3.576. The paper identifies DressCode as the strongest evidence for state-of-the-art performance because the method handles upper-body clothes, lower-body clothes, and full-body dresses within a single mask-free framework.

The ablation findings summarized in the paper attribute performance gains to four factors. First, positional encoding matters: the ablation “w/o Pos Encoding” worsens FID and KID and weakens background or structure alignment. Second, full-parameter update matters: “w/o Full-Params” performs worse than the full model. Third, masked attention helps by improving feature robustness, output coherence, and computational efficiency. Fourth, the conditional QKV branches are important for adding control capacity, preserving backbone behavior, and enabling direct garment/person fusion.

Human evaluation compares Kling, OutfitAnyone, and JCo-MVTON across four scenarios—Overall, Upper, Lower, and Dress—and five metrics: Image Authenticity, Image Clarity, Silhouette Consistency, Detail Consistency, and Overall Harmony. The reported conclusion is that JCo-MVTON is rated best overall, especially on authenticity, clarity, harmony, and silhouette consistency. The paper notes a minor exception: Kling can slightly outperform on Detail Consistency in the dress scenario, while JCo-MVTON remains strongest in overall harmony and most other dimensions.

The paper also claims strong generalization on in-the-wild images, anime-style imagery, fashion or e-commerce images, upper/lower/dress categories, and commercial benchmark comparisons. It compares against systems including OutfitAnyone, Kling, and GPT-4o and reports more stable, higher-fidelity results. Because these comparisons are described at a high level in the provided summary, the safest interpretation is that the paper positions JCo-MVTON as extending beyond curated academic benchmarks into commercial-style scenarios.

The limitations mentioned or implied are equally specific. Fine details such as hand gestures, small accessories, and some complex poses remain challenging. High-quality data curation is expensive and iterative. The method is compute-heavy because of diffusion-transformer training and high-resolution generation. Some scenarios still show slight instability compared with the best commercial systems in specific detail metrics. These caveats delimit the paper’s claims: JCo-MVTON is presented not as a complete solution to virtual try-on, but as a mask-free framework in which architectural conditioning and bidirectional dataset bootstrapping jointly improve controllability, fidelity, and generalization.