Papers
Topics
Authors
Recent
Search
2000 character limit reached

OmniVTON++: Training-Free Universal VTON

Updated 4 July 2026
  • The paper introduces OmniVTON++, a training-free universal VTON framework that refines garment alignment and pose regulation using a modular design to improve metrics such as FID and SSIM.
  • OmniVTON++ employs a three-module pipeline—Structured Garment Morphing for precise garment alignment, Principal Pose Guidance for sustained pose consistency, and Continuous Boundary Stitching for seamless boundary blending.
  • The method operates without retraining by leveraging off-the-shelf components, enabling versatile application across single/multi-garment, multi-human, and anime character try-on scenarios.

Searching arXiv for the specified paper and closely related work to ground the article in the cited literature. OmniVTON++ is a training-free image-based virtual try-on (VTON) framework introduced in “OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidance” (Yang et al., 16 Feb 2026). It is designed for universal applicability across heterogeneous VTON conditions without task-specific retraining, and addresses three coupled failure modes in prior systems: garment alignment, human structural coherence, and boundary continuity. The framework coordinates Structured Garment Morphing (SGM), Principal Pose Guidance (PPG), and Continuous Boundary Stitching (CBS) into a single inference pipeline that remains compatible with different diffusion backbones and supports single-garment, multi-garment, single-human, multi-human, and anime character virtual try-on scenarios (Yang et al., 16 Feb 2026).

1. Position within training-free and unified VTON research

OmniVTON++ is situated within a line of work that seeks to remove dataset- or task-specific retraining from VTON deployment. The paper states that most image-based VTON models are trained for a specific regime, such as in-shop “Shop-to-Model” or in-the-wild “StreetTryOn,” and therefore rely on paired supervision or domain-specific priors. It further notes that Thin-Plate-Spline (TPS) or learned flow modules often fail under large pose variations or when the garment input type shifts, and that pose or parsing conditions are usually injected during training in ways that are not easily transferable across architectures such as U-Net and DiT (Yang et al., 16 Feb 2026).

The immediate precursor is OmniVTON, which is described as the first training-free universal VTON framework and also centers on the decoupling of garment appearance and body pose (Yang et al., 20 Jul 2025). OmniVTON++ preserves the training-free objective but reformulates pose control: instead of DDIM inversion with Spectral Pose Injection, it introduces Principal Pose Guidance as a step-wise structural regulator during diffusion sampling (Yang et al., 16 Feb 2026). This suggests a shift from frequency-domain pose preservation toward latent-space guidance driven by a proxy representation and principal-subspace selection.

A related but methodologically distinct development is OmniDiT, a unified mask-free Diffusion Transformer for model-based VTON, model-free VTON, and VTOFF that relies on training, a large curated dataset, token concatenation, adaptive positional encoding, Shifted Window Attention, and task-specific objectives (Zeng et al., 20 Mar 2026). In contrast, OmniVTON++ retains frozen off-the-shelf components and emphasizes universality through inference-time composition rather than learned unification (Yang et al., 16 Feb 2026).

2. Architectural composition and design objectives

OmniVTON++ consists of three linked modules: Structured Garment Morphing, Principal Pose Guidance, and Continuous Boundary Stitching. The framework requires no additional training and uses only off-the-shelf parsing, pose, diffusion, and dressing models (Yang et al., 16 Feb 2026). Its stated objectives are to operate “out-of-the-box” on arbitrary garment and person inputs, handle flat-lay and person-worn garments, support upper-body, lower-body, dresses, multi-garment, multi-human, and anime inputs, preserve fine-grained texture through explicit correspondence-driven warping, maintain persistent pose control during diffusion sampling without over-regularizing garment appearance, and produce seamless boundary blending across part regions (Yang et al., 16 Feb 2026).

The three modules divide the problem by function. SGM constructs a coarse garment prior aligned to the target body geometry. PPG enforces target-pose consistency during denoising while leaving residual modes available for garment appearance. CBS removes seam artifacts by fusing garment-stream and person-stream features inside attention layers (Yang et al., 16 Feb 2026). This modular factorization continues the decoupling logic already present in OmniVTON, where garment and pose were handled as distinct constraints (Yang et al., 20 Jul 2025), but OmniVTON++ extends that principle to a more explicitly staged pipeline.

A plausible implication is that the design targets portability across backbone families by avoiding training-time entanglement between condition encoding and backbone internals. The paper states explicitly that the method operates across scenarios and diffusion backbones within a single formulation (Yang et al., 16 Feb 2026).

3. Structured Garment Morphing

Structured Garment Morphing is the module responsible for correspondence-driven garment adaptation. Its goal is to warp a source garment image IcI_c so that it matches the target person’s body geometry in IpI_p, producing a coarse prior IwI_w that retains texture detail (Yang et al., 16 Feb 2026).

If the garment input is flat-lay, OmniVTON++ first synthesizes a pseudo-person image IoI_o wearing IcI_c in A-pose via a pretrained virtual-dressing module. The module then uses human parsing maps Pp,PoP_p, P_o, garment masks Ms,MoM_s, M_o, and keypoint detections {Bpi},{Boi}\{B_p^i\}, \{B_o^i\} (Yang et al., 16 Feb 2026). For each semantic part i∈{1,…,5}i \in \{1,\dots,5\}, such as torso and upper or lower arms, it defines a part support indicator:

IRegioni(x,y)={1,(x,y)∈Poi∩Mo∩Boi,0,otherwise.\mathbb{I}_{\text{Region}_i}(x,y)= \begin{cases} 1, & (x,y)\in P_o^i \cap M_o \cap B_o^i,\\ 0, & \text{otherwise.} \end{cases}

Localized homographies Ho→pi∈R3×3\mathcal{H}_{o\to p}^i \in \mathbb{R}^{3\times 3} are then estimated by optimizing

min⁡H∑(x,y)∈Regioni∥H [x,y,1]⊤−[xp,yp,1]⊤∥2\min_{\mathcal H}\sum_{(x,y)\in\text{Region}_i}\|\mathcal H\,[x,y,1]^\top - [x_p,y_p,1]^\top\|^2

via Levenberg–Marquardt (Yang et al., 16 Feb 2026).

The warped garment prior is assembled piecewise:

[x′,y′,1]⊤=∑iIRegioni(x,y)  Ho→pi [x,y,1]⊤,[x',y',1]^\top = \sum_i \mathbb{I}_{\text{Region}_i}(x,y)\; \mathcal{H}_{o\to p}^i\,[x,y,1]^\top,

after which occluded pixels are masked out using the part segmentation PpP_p (Yang et al., 16 Feb 2026). The paper also provides pseudocode specifying the sequence: generate IoI_o when unavailable, parse IoI_o and IpI_p, obtain garment masks, detect keypoints, estimate part-wise homographies, and write warped pixels into IwI_w if the part label matches PpP_p (Yang et al., 16 Feb 2026).

Compared with OmniVTON, which already used skeleton-guided, multi-part piecewise homography for garment prior generation, OmniVTON++ makes the correspondence structure more explicit by defining supported regions with parsing, masking, and keypoint-derived spatial support (Yang et al., 20 Jul 2025, Yang et al., 16 Feb 2026). This suggests a refinement from coarse body-part warping toward support-aware local alignment.

4. Principal Pose Guidance

Principal Pose Guidance is the principal methodological distinction of OmniVTON++ relative to OmniVTON. Its goal is to enforce the target human pose extracted from IpI_p during diffusion sampling while allowing garment appearance to evolve freely (Yang et al., 16 Feb 2026).

The method begins by constructing a proxy image IproxyI_{\text{proxy}} that preserves pose but removes original clothing detail. This is done by inpainting background under the garment mask MsM_s, filling body-occluded pixels Mb=Md∩MsM_b = M_d \cap M_s with average skin color chc_h, setting the garment region MtM_t to constant color ctc_t, and keeping the rest unchanged from IpI_p. The proxy image is then encoded into a codebook latent zproxyz_{\text{proxy}} (Yang et al., 16 Feb 2026).

Sampling follows DDCM, cited in the paper as DDCM [Ohayon et al. ICML 2025], with latent update

$z_{t-1} = \sqrt{\alpha_{t-1}\;\mu_\theta(z_t,t,c) + \sqrt{1-\alpha_{t-1}-\sigma_t^2}\;\epsilon_\theta(z_t,t,c) + \sigma_t\;C_t(k_t),$

where the discrete noise index ktk_t is chosen by inner-product alignment (Yang et al., 16 Feb 2026). OmniVTON++ restricts pose guidance to the principal subspace of the intermediate prediction zˉ0∣t\bar z_{0|t}. Writing zˉ0∣t=PCA(zˉ0∣t)+residual\bar z_{0|t}=\mathrm{PCA}(\bar z_{0|t})+\text{residual}, the method selects

$k_t=\arg\max_{k\in\{1..K\}} \Big\langle C_t(k),\,z_{\mathrm{proxy}-\bar z_{0|t}^{\mathrm{PCA}\Big\rangle.$

The paper states that this enforces pose consistency while leaving residual modes to garment and texture (Yang et al., 16 Feb 2026).

The method also provides an implicit guidance interpretation:

$\mathcal L_{\mathrm{PPG}(z_t)= -\langle \epsilon,\;z_{\mathrm{proxy}-\bar z_{0|t}^{\mathrm{PCA}\rangle,$

maximized over discrete noise candidates ϵ=Ct(k)\epsilon=C_t(k), although no explicit loss is minimized at inference (Yang et al., 16 Feb 2026).

Relative to OmniVTON’s DDIM inversion with Spectral Pose Injection, which retained low-frequency components of an inverted latent and replaced high-frequency components with fresh noise, PPG performs pose regulation throughout diffusion via discrete noise selection within a principal subspace (Yang et al., 20 Jul 2025, Yang et al., 16 Feb 2026). This suggests a more persistent and temporally distributed form of structural control than one-shot latent initialization.

5. Continuous Boundary Stitching and backbone compatibility

Continuous Boundary Stitching is the module that removes seam artifacts produced by part-wise garment morphing. In OmniVTON++, CBS fuses garment-stream and person-stream features through cross-attention (Yang et al., 16 Feb 2026). The person stream features Qp′,Kp′,Vp′Q_p', K_p', V_p' attend over concatenated keys [Kp′∥Kc][K_p' \parallel K_c] and values [Vp′∥(Vc⊙D(Mc))][V_p' \parallel (V_c \odot D(M_c))]:

fp′=Softmax(Qp′[Kp′∥Kc]⊤d)  [Vp′∥(Vc⋅D(Mc))].f_p'=\mathrm{Softmax}\Big(\tfrac{Q_p'[K_p'\parallel K_c]^\top}{\sqrt d}\Big)\; [V_p'\parallel (V_c\cdot D(M_c))].

The garment stream attends similarly, but aggregates only its own values:

Ac=Softmax(Qc[Kc∥Kp′]⊤d),fc=Ac[:,1:n]  Vc.A_c=\mathrm{Softmax}\Big(\tfrac{Q_c[K_c\parallel K_p']^\top}{\sqrt d}\Big),\quad f_c=A_c[:,1:n]\;V_c.

The paper states that this forces boundary pixels to access both garment and person context in the same self-attention, which suffices to smooth seams without any extra per-pixel loss (Yang et al., 16 Feb 2026).

For DiT backbones, OmniVTON++ applies Positional Index Realignment so that each input’s tokens occupy disjoint RoPE index ranges (Yang et al., 16 Feb 2026). This is notable because backbone heterogeneity is one of the stated limitations of prior pose and parsing conditioning strategies. OmniDiT also addresses token interaction and positional conflict in DiT through adaptive positional encoding, but it does so in a fully trained omni-VTON transformer rather than a training-free inference framework (Zeng et al., 20 Mar 2026). The two approaches therefore address similar transformer-conditioning constraints at different levels: OmniDiT through learned token-space design, OmniVTON++ through inference-time token-index realignment.

The boundary-stitching idea descends from OmniVTON’s CBS module, which used bidirectional cross-path attention modulation in U-Net self-attention layers to blend the garment prior with the cloth-infused stream (Yang et al., 20 Jul 2025). OmniVTON++ preserves the same basic purpose but formalizes it as a dual-stream feature interaction that is explicitly portable to both U-Net and DiT variants (Yang et al., 16 Feb 2026).

6. Unified inference pipeline, supported settings, and implementation

The paper describes a one-pass inference procedure. In Step 1, SGM generates or loads IoI_o for Shop-to-X settings, parses images, detects keypoints, computes homographies, assembles the coarse warp IwI_w, and injects IwI_w into the person image to produce a garment-infused image Ip′I_p' (Yang et al., 16 Feb 2026). In Step 2, the proxy image IproxyI_{\text{proxy}} is built through the four-stage composition described in Eqs. 5–11. In Step 3, diffusion sampling proceeds by encoding IproxyI_{\text{proxy}} to zproxyz_{\text{proxy}}, initializing noise zTz_T, repeatedly predicting zˉ0∣t\bar z_{0|t}, applying PCA, using Eq. (12) to choose ktk_t, updating zt−1z_{t-1} via Eq. (13), and keeping cross-stream attention active through CBS or CBS-DiT, before decoding z0z_0 to the final try-on image (Yang et al., 16 Feb 2026). No module requires weight updates or fine-tuning; parsers, pose estimators, dressing models, and diffusion models remain frozen (Yang et al., 16 Feb 2026).

The framework supports several extended scenarios. For multi-garment try-on, several Io(j)I_o^{(j)} are spatially concatenated to feed the garment stream, each is morphed independently, and all are injected together into IpI_p (Yang et al., 16 Feb 2026). For multi-human try-on, SGM is run per person instance and the warped patches are merged into the cloth-agnostic mask, while CBS/CBS-DiT and PPG remain unchanged (Yang et al., 16 Feb 2026). For anime character try-on, the same pipeline is directly applied to stylized person images, with the paper stating that results preserve character identity and garment fidelity (Yang et al., 16 Feb 2026). These extensions align with the broader “universal garment representation” objective described in the paper.

The implementation details reported are specific. The backbones are Stable Diffusion v2.0 with U-Net inpainting and DDIM 50 steps, and FLUX.1 Fill with DPM-Solver++ SDE 30 steps (Yang et al., 16 Feb 2026). PPG uses codebook size K=64K=64 and the top 3 principal components at each step. The virtual-dressing module is IMAGDressing-v1 default. Parsing uses TAPPS for body parts and PGN for garments, OpenPose for keypoints, and SAM/Navier-Stokes for fallback inpainting (Yang et al., 16 Feb 2026). The reported hardware is a single NVIDIA RTX A6000, with runtime of approximately 1.2 s1.2\,\mathrm{s} per image on SD-2.0 at 512×384512\times 384 and 50 steps, and approximately 0.9 s0.9\,\mathrm{s} on FLUX with 30 steps (Yang et al., 16 Feb 2026).

7. Experimental evaluation and relation to adjacent frameworks

OmniVTON++ is evaluated in cross-dataset and cross-garment-type settings without pre-training any module on the target VTON benchmarks (Yang et al., 16 Feb 2026). The reported metrics include FIDu\mathrm{FID}_u, FIDp\mathrm{FID}_p, SSIMp\mathrm{SSIM}_p, and LPIPSp\mathrm{LPIPS}_p (Yang et al., 16 Feb 2026). On VITON-HD in the Shop-to-Model unpaired setting, OmniVTON++ with SD-2.0 reports FIDu=9.19\mathrm{FID}_u = 9.19, SSIMp=0.843\mathrm{SSIM}_p = 0.843, and LPIPSp=0.130\mathrm{LPIPS}_p = 0.130, while OmniVTON++ with FLUX reports FIDu=9.29\mathrm{FID}_u = 9.29, SSIMp=0.849\mathrm{SSIM}_p = 0.849, and LPIPSp=0.121\mathrm{LPIPS}_p = 0.121 (Yang et al., 16 Feb 2026). The paper states that these variants outperform GP-VTON, CAT-DM, D⁴-VTON, IDM-VTON, OOTDiffusion, and Any2AnyTryOn on that benchmark (Yang et al., 16 Feb 2026). On DressCode, the framework is reported as top-2 on all upper, lower, and dress categories, and on StreetTryOn as best or second-best across Shop-to-Street, Model-to-Model, Model-to-Street, and Street-to-Street (Yang et al., 16 Feb 2026).

The ablation study attributes substantial performance changes to each module. Removing SGM raises FID\mathrm{FID} from $9.19$ to $18.45$. Replacing PPG with SPI or ControlNet causes SSIM\mathrm{SSIM} to drop by more than $0.01$ and worsens FID\mathrm{FID} by $0.2$ to $1.3$. Turning off CBS or CBS-DiT produces clear seam artifacts, raises FID\mathrm{FID} by $2$ to $3$ points, and increases LPIPS\mathrm{LPIPS} by $0.02$ (Yang et al., 16 Feb 2026). These results support the paper’s claim that garment alignment, structural regulation, and seam handling are interdependent rather than separable post hoc corrections.

A compact comparison with the adjacent OmniVTON and OmniDiT lines is informative:

Framework Core regime Key pose mechanism
OmniVTON (Yang et al., 20 Jul 2025) Training-free universal VTON DDIM inversion with Spectral Pose Injection
OmniVTON++ (Yang et al., 16 Feb 2026) Training-free universal VTON Principal Pose Guidance during sampling
OmniDiT (Zeng et al., 20 Mar 2026) Trained unified VTON/VTOFF DiT Flow-matching DiT with multi-condition token concatenation

OmniVTON reports on VITON-HD FIDu=9.621\mathrm{FID}_u = 9.621, FIDp=7.758\mathrm{FID}_p = 7.758, SSIMp=0.832\mathrm{SSIM}_p = 0.832, and LPIPSp=0.145\mathrm{LPIPS}_p = 0.145, and its ablations show gains from SGM, CBS, and SPI relative to a text-only base (Yang et al., 20 Jul 2025). OmniVTON++ improves the reported VITON-HD unpaired values to FIDu=9.19\mathrm{FID}_u = 9.19 and SSIMp=0.843\mathrm{SSIM}_p = 0.843 with SD-2.0, with lower LPIPSp=0.130\mathrm{LPIPS}_p = 0.130, and to SSIMp=0.849\mathrm{SSIM}_p = 0.849 and LPIPSp=0.121\mathrm{LPIPS}_p = 0.121 with FLUX (Yang et al., 16 Feb 2026). OmniDiT, by contrast, reports model-based VITON-HD results of FID=6.46\mathrm{FID}=6.46, KID=0.75×10−3\mathrm{KID}=0.75\times 10^{-3}, SSIM=0.8838\mathrm{SSIM}=0.8838, and LPIPS=0.0784\mathrm{LPIPS}=0.0784, but it does so within a trained framework supported by the Omni-TryOn dataset and additional losses (Zeng et al., 20 Mar 2026). This comparison should not be read as a direct ranking across identical settings; the regimes are different. A plausible implication is that OmniVTON++ occupies a distinct methodological niche: maximizing deployment universality without retraining, rather than maximizing absolute performance within a training-based unified model.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to OmniVTON++.