OmniVTON: Universal Virtual Try-On Framework

• OmniVTON is a training-free virtual try-on framework that decouples garment and pose conditioning using a diffusion-based formulation.
• It employs geometrically aligned garment priors, continuous boundary stitching, and spectral pose injection to synthesize realistic person imagery.
• Extensions like OmniVTON++ and OmniDiT refine pose guidance and integrate transformer architectures for unified virtual try-on and try-off tasks.

Searching arXiv for the OmniVTON paper family and closely related entries to ground the article in the latest available preprints. OmniVTON is a training-free universal virtual try-on framework for image-based garment transfer that was introduced to unify supervised in-shop and unsupervised in-the-wild settings within a single diffusion-based formulation. Its central design principle is to decouple garment conditioning and pose conditioning: garment appearance is injected through a geometrically aligned garment prior and boundary-aware inpainting, whereas pose is injected through DDIM inversion and spectral manipulation of latent noise. Subsequent work extends this line in two directions: OmniVTON++ replaces single-step pose injection with step-wise pose guidance and refines the boundary module for both U-Net and DiT backbones, while OmniDiT moves to a trained, unified Diffusion Transformer that combines model-based VTON, model-free VTON, and VTOFF in one model (Yang et al., 20 Jul 2025, Yang et al., 16 Feb 2026, Zeng et al., 20 Mar 2026).

1. Problem formulation and conceptual basis

OmniVTON addresses image-based Virtual Try-On (VTON), where the task is to synthesize realistic person imagery through garment re-rendering under human pose and body constraints. The original formulation was motivated by a split in the literature between supervised in-shop approaches, which ensure high fidelity but struggle with cross-domain generalization, and unsupervised in-the-wild methods, which improve adaptability but remain constrained by data biases and limited universality (Yang et al., 20 Jul 2025).

The framework is explicitly training-free. It does not retrain the diffusion backbone for each dataset or task condition, and instead composes pretrained components with geometric warping, frequency-domain pose conditioning, and attention-level feature fusion. This design is intended to avoid the bias that arises when diffusion models handle multiple conditions simultaneously. In the original method, garment guidance enters through the input image $I_p'$ and mask, while pose guidance enters exclusively through the mixed latent noise $\hat z_T$ (Yang et al., 20 Jul 2025).

A later extension, OmniVTON++, retains the training-free premise but reorganizes the pipeline into three named modules: Structured Garment Morphing (SGM), Principal Pose Guidance (PPG), and Continuous Boundary Stitching (CBS), all wrapped around an off-the-shelf diffusion inpainting backbone such as Stable Diffusion or FLUX. No retraining is performed; all modules rely on pretrained tools, including a human parser, pose estimator, and diffusion model (Yang et al., 16 Feb 2026).

This suggests that OmniVTON is best understood not as a single architecture in the narrow sense, but as a design pattern for disentangling garment geometry, human structure, and diffusion synthesis.

2. Original OmniVTON pipeline

The original OmniVTON pipeline begins with two inputs: a source garment image $I_c$, which may be standalone or worn, and a target person image $I_p$. The first stage constructs a garment prior; the second stage performs inpainting with decoupled pose conditioning and continuous boundary stitching (Yang et al., 20 Jul 2025).

When only a standalone garment image is available, OmniVTON first generates a pseudo-person image $I_o$ that “wears” the garment. This is done by cross-attention in a pretrained diffusion U-Net. Let $z_t$ denote diffusion noise at timestep $t$, $\tilde M_c$ the inverted cloth mask on $I_c$, $I_m$ a cloth-agnostic version of the real person, and $\hat z_T$0 the agnostic mask on $\hat z_T$1. Two parallel denoising streams are run, and the wearer’s semantics are injected into the garment stream by

$\hat z_T$2

Semantic correspondence is then established by running OpenPose to obtain 25 keypoints and grouping them into $\hat z_T$3 semantic regions. For each region $\hat z_T$4, a bounding box $\hat z_T$5 is defined on $\hat z_T$6 and $\hat z_T$7 on $\hat z_T$8, and a part-segmentation map $\hat z_T$9 refines the support: $I_c$0

Localized piecewise homography is then used to align the garment to the target body. For each region $I_c$1, a $I_c$2 homography $I_c$3 is solved by Levenberg–Marquardt, and the pixels of $I_c$4 are mapped by

$I_c$5

The output $I_c$6 is a coarse aligned garment prior. Because the warping is region-wise, seams can appear at region boundaries; this motivates the next stage (Yang et al., 20 Jul 2025).

In the inpainting stage, a garment-infused image is formed as

$I_c$7

The pretrained U-Net itself is unchanged. The explicitly training-free components are the homography solver, the Fourier-based pose conditioning, and attention calls to key/value buffers (Yang et al., 20 Jul 2025).

3. Garment prior generation and boundary refinement

In OmniVTON terminology, the garment prior is a coarse but texture-faithful deformation of the source garment. Its purpose is to preserve garment details under substantial spatial deformation. The original paper divides this into pseudo-person generation, semantic correspondence, and localized homography, collectively yielding a body-aligned prior $I_c$8 (Yang et al., 20 Jul 2025).

To remove seams introduced by stitching across warped regions, OmniVTON uses Continuous Boundary Stitching (CBS) during the final inpainting stage. CBS modulates self-attention in each transformer block using both the garment-infused person image $I_c$9 and the source garment $I_p$0. The $I_p$1-path enhancement is

$I_p$2

where $I_p$3 is the low-resolution downsampled mask. The $I_p$4-path refinement is

$I_p$5

The stated role of CBS is to erase multi-region seams and blend fine texture details. By back-and-forth attention mixing, it removes hard seams while preserving cloth texture (Yang et al., 20 Jul 2025).

OmniVTON++ retains the same basic concern but re-specifies the garment module as Structured Garment Morphing. When the garment image is flat-lay and keypoint detection and parsing fail, a pseudo-person $I_p$6 is synthesized by feeding $I_p$7 through a pretrained garment-centric dressing module, IMAGDressing-v1, together with a fixed “A-pose” keypoint set and a generic prompt, “white background, no occlusion” (Yang et al., 16 Feb 2026). It then defines effective supports by

$I_p$8

and assembles the warped prior $I_p$9 through localized homographies

$I_o$0

In OmniVTON++, CBS is also generalized. For U-Net backbones, dual-stream attention refines boundaries by mixing person and garment streams asymmetrically; for DiT backbones, CBS-DiT uses Positional Index Realignment so that tokens from text, person, and garment receive disjoint rotary-encoding index ranges rather than colliding indices (Yang et al., 16 Feb 2026).

4. Pose conditioning and diffusion dynamics

The most distinctive element of the original OmniVTON is its decoupled pose-conditioning strategy. Rather than encode garment and pose in the same pathway, it obtains target pose from DDIM inversion of the person image $I_o$1 to a latent noise $I_o$2, then suppresses texture interference by retaining only the low-frequency components of the inverted noise (Yang et al., 20 Jul 2025).

The method first computes

$I_o$3

then fuses them with a Gaussian mask

$I_o$4

and reconstructs

$I_o$5

This mechanism is called Spectral Pose Injection (SPI). Its stated effect is to preserve coarse silhouette and pose while re-randomizing high-frequency clothing cues (Yang et al., 20 Jul 2025).

Sampling then proceeds with garment guidance in $I_o$6 and pose guidance in $I_o$7. The one-step update is written as

$I_o$8

The paper’s interpretation is that because pose lives in the latent noise and garment in the image hints, neither condition “overpowers” the other (Yang et al., 20 Jul 2025).

OmniVTON++ replaces this single-initialization strategy with Principal Pose Guidance. It operates within the Discrete Diffusion Codebook Model framework, where at timestep $I_o$9,

$z_t$0

Because the target image $z_t$1 is unavailable, the method builds a proxy image $z_t$2 containing pose information but not the original garment texture. The proxy is formed by ordered region composition: background recovery with inpainting over the garment mask, body completion using average skin color $z_t$3, target-garment cues colored with average garment color $z_t$4, and preserved regions copied from $z_t$5 (Yang et al., 16 Feb 2026).

At each diffusion step, OmniVTON++ performs Pose-Guided Noise Selection: $z_t$6 where $z_t$7 retains only the top few principal components of $z_t$8. The update becomes

$z_t$9

The stated purpose is to steer the sampling subspace along pose dimensions while leaving orthogonal components free to model garment texture and lighting (Yang et al., 16 Feb 2026).

5. Empirical performance and demonstrated scope

The original OmniVTON was evaluated on VITON-HD, DressCode, and StreetTryOn under paired and unpaired settings using FID, SSIM, and LPIPS. On VITON-HD, with model checkpoints pre-trained on DressCode, it reports $t$0, $t$1, $t$2, and $t$3, compared with $t$4, $t$5, $t$6, and $t$7 for IDM-VTON. On DressCode, with VITON-HD pre-trained models, it reports $t$8, $t$9, $\tilde M_c$0, and $\tilde M_c$1, compared with $\tilde M_c$2, $\tilde M_c$3, $\tilde M_c$4, and $\tilde M_c$5 for IDM-VTON. On StreetTryOn, the four cross-scenario FID scores are Shop→Street $\tilde M_c$6, Model→Model $\tilde M_c$7, Model→Street $\tilde M_c$8, and Street→Street $\tilde M_c$9 (Yang et al., 20 Jul 2025).

A notable capability claim in the original paper is that OmniVTON is the first framework capable of multi-human VTON. The described mechanism is horizontal concatenation of multiple $I_c$0, generation of multiple pseudo-persons, and independent warping for each (Yang et al., 20 Jul 2025).

The ablation study on VITON-HD isolates the contribution of the three core modules. The base text-only system reports $I_c$1, $I_c$2, $I_c$3, and $I_c$4 for $I_c$5, $I_c$6, $I_c$7, and $I_c$8. Adding SGM only yields $I_c$9, $I_m$0, $I_m$1, and $I_m$2. Adding SGM+CBS yields $I_m$3, $I_m$4, $I_m$5, and $I_m$6. SPI only gives $I_m$7, $I_m$8, $I_m$9, and $\hat z_T$00. The full OmniVTON gives $\hat z_T$01, $\hat z_T$02, $\hat z_T$03, and $\hat z_T$04 (Yang et al., 20 Jul 2025).

OmniVTON++ evaluates cross-dataset and cross-garment-type generalization using unpaired FID and, where ground truth exists, paired FID, SSIM, and LPIPS. The paper states that in every setting, OmniVTON++ in both SD-2.0 and FLUX variants ranks first or second on FID, SSIM, and LPIPS, often improving over the original OmniVTON by $\hat z_T$05–$\hat z_T$06. Its ablation study reports, for example, that adding SGM to a vanilla diffusion base cuts unpaired FID on VITON-HD SD-2.0 from $\hat z_T$07 to $\hat z_T$08; PPG further reduces FID to $\hat z_T$09 and raises SSIM from $\hat z_T$10 to $\hat z_T$11; adding CBS then yields $\hat z_T$12 FID and $\hat z_T$13 SSIM; and all three modules together yield $\hat z_T$14 FID and $\hat z_T$15 SSIM (Yang et al., 16 Feb 2026).

Within the broader omni-VTON/VTOFF problem, OmniDiT reports results on model-based VTON, model-free VTON, and Try-Off. On VITON-HD model-based VTON, it reports $\hat z_T$16, $\hat z_T$17, $\hat z_T$18, and $\hat z_T$19; on DressCode, $\hat z_T$20, $\hat z_T$21, $\hat z_T$22, and $\hat z_T$23; and on Omni-TryOn, $\hat z_T$24, $\hat z_T$25, $\hat z_T$26, and $\hat z_T$27. For model-free VTON on VITON-HD, it reports $\hat z_T$28, $\hat z_T$29, $\hat z_T$30, and $\hat z_T$31. For Try-Off on VITON-HD, it reports $\hat z_T$32, $\hat z_T$33, $\hat z_T$34, $\hat z_T$35, and $\hat z_T$36 (Zeng et al., 20 Mar 2026).

6. Extensions, unification, and limitations

OmniVTON++ is presented as an extension over the ICCV ’25 OmniVTON. Relative to the original system, SGM replaces attention-modulated outpainting with a garment-centric virtual dressing module for more reliable pseudo-person geometry; PPG replaces single-step Spectral Pose Injection with step-wise pose guidance via codebook noise selection on principal components; CBS is extended to DiT backbones via Positional Index Realignment, yielding CBS-DiT; and the evaluation adds robustness tests across backbones and scenarios as well as multi-garment, multi-human, and anime character virtual try-on (Yang et al., 16 Feb 2026).

A separate line of development is OmniDiT, which is not training-free. It proposes an omni Virtual Try-On framework based on the Diffusion Transformer and combines try-on and try-off tasks into one unified model. The method builds a large VTON dataset, Omni-TryOn, through a self-evolving curation pipeline. The dataset contains $\hat z_T$37 samples, approximately $\hat z_T$38 unique garments, spans $\hat z_T$39 categories, and holds out $\hat z_T$40 stratified samples for testing. Its architecture concatenates text tokens, noisy image latent tokens, and multiple reference-image tokens into a single sequence,

$\hat z_T$41

augmented with adaptive positional encoding using “3D RoPE-style” position indices $\hat z_T$42 (Zeng et al., 20 Mar 2026).

To control the computational cost of long token sequences, OmniDiT introduces Shifted Window Attention into the diffusion model. The paper states that global self-attention over a sequence of length $\hat z_T$43 costs $\hat z_T$44, whereas the shifted-window formulation reduces this to $\hat z_T$45 per layer. On an A800 with two $\hat z_T$46 inputs and a $\hat z_T$47 output, SWA cuts inference time from $\hat z_T$48 to $\hat z_T$49, a reduction of $\hat z_T$50 (Zeng et al., 20 Mar 2026). OmniDiT also introduces multiple timestep prediction and an alignment loss

$\hat z_T$51

combined as

$\hat z_T$52

The reported ablation shows that MTP with $\hat z_T$53 lowers FID from $\hat z_T$54 to $\hat z_T$55 on VITON-HD, and adding alignment further lowers FID to $\hat z_T$56, with $\hat z_T$57 and $\hat z_T$58 (Zeng et al., 20 Mar 2026).

The limitations explicitly documented for OmniVTON++ remain relevant to the training-free line. The method depends on external analysis tools: misdetected keypoints or failed parsing or DensePose can impair SGM and PPG. Accessories such as necklaces and hats may be mistakenly masked and inpainted over; the paper suggests that one may mitigate this by incorporating a segmentation model such as SAM 3 to exclude such regions. The approach is also currently limited to single-view images, and true 3D consistency or video stability remains open (Yang et al., 16 Feb 2026).

A common interpretation of the sequence from OmniVTON to OmniVTON++ and OmniDiT is that the field has explored two complementary routes to universality. One route preserves the training-free premise and improves conditioning fidelity through better geometry, step-wise pose control, and boundary-aware attention. The other route adopts a unified trained transformer with large-scale data curation and multi-task supervision.