- The paper introduces a 5B-parameter MMDiT virtual try-on system that conditions on up to six references across eight fashion categories while preserving identity, backgrounds, and garment details.
- Tstars-Tryon 1.0 achieves overall scores of 9.372 for single-garment and 9.171 for multi-garment evaluation, outperforming academic and general-purpose editing baselines on the Tstars-VTON benchmark.
- The system combines automated data creation, staged training, reinforcement learning, and distillation to reach 3.92-second single-item and 6.74-second multi-item inference on an H200, supporting millions of Taobao users.
Tstars-Tryon 1.0 is a commercial-scale virtual try-on system developed by the Pailitao team at Taobao/Alibaba, designed to close the gap between academic try-on research and industrial deployment requirements. The paper argues that existing methods—both specialized academic models and general-purpose image editing systems—fail simultaneously on four axes that commercial deployment demands: robustness to in-the-wild inputs, photorealistic garment fidelity, flexible multi-item composition, and near real-time inference latency. The system is deployed as the "AI Try-On" service on the Taobao App, serving millions of users and tens of millions of requests.
System design
The system is built around a unified MMDiT architecture (2604.19748), treating virtual try-on as a specialized image editing task rather than a traditional inpainting problem. This choice allows simultaneous conditioning on multiple reference images—up to 6—across 8 fashion categories (tops, pants, skirts, dresses, coats, shoes, bags, hats), with coordinated control over person identity and background. The primary DiT model is deliberately kept at 5B parameters to control serving cost.
Several supporting components complete the pipeline:
- Data engine: an automated pipeline combining element decomposition, retrieval-based recall, customized captioners, knowledge-enhanced VLM post-filtering, and perceptual metric screening to produce large-scale multi-item editing data.
- Training infrastructure: native support for variable resolutions and arbitrary reference counts, using Data Parallelism, Tensor Parallelism, and Data Packing adapted from NaViT-style patch packing (2604.19748) to eliminate bucketing waste.
- Multi-stage training: pre-training with task- and content-balanced data under progressive difficulty scaling; progressive-resolution continuous training; curated SFT; and reinforcement learning via DiffusionNFT (2604.19748) with group-level trajectory sampling and multi-dimensional group-relative rewards.
- Prompt enhancement: a tailored rewriter model for precise semantic guidance of complex editing operations.
- Inference acceleration: CFG distillation combined with step distillation yields 3.92 s latency for single-garment and 6.74 s for multi-garment try-on (5 reference images on average) on an H200 GPU. By comparison, the paper reports ~200 s for top open-source baselines (QwenEdit-2511, FLUX.2-dev)—a roughly 50× speedup, though closed-source API timings are noted as possibly including network latency.
Tstars-VTON Benchmark
A central contribution is the Tstars-VTON Benchmark, motivated by concrete deficiencies in VITON-HD and DressCode: homogeneous studio backgrounds, restricted categories limited to upper/lower/dress bodies, single-garment-only protocols, and the implicit assumption that reference garments are pristine flat-lay images. The new benchmark contains 1,780 paired samples spanning 5 garment and 3 accessory categories, 465 fine-grained subcategories, and 1–6 layered items per sample. It includes 29.6% complex poses versus only 8.2% simple poses, over 40% highly complex backgrounds, and applies face swapping against a licensed surrogate database for privacy preservation. It supports fully unpaired evaluation by decoupling the person and garment databases.
Evaluation uses a two-stage VLM-driven protocol scored on 1–10 Likert scales across four dimensions: Identity Consistency and Garment Fidelity (garment-aware stage), and Background Preservation plus Physical and Structural Logic (garment-agnostic stage). The Overall Score is the geometric mean of the four dimensions, chosen explicitly because it penalizes weak links—a defensible design choice given that FID-style metrics fail to capture fine-grained defects such as fabric interpenetration or anatomical anomalies.
Quantitative results
On single-garment evaluation, Tstars-Tryon 1.0 achieves an overall score of 9.372, ahead of Seedream5 lite (9.301), Nano Banana Pro (9.229), GPT-Image-1.5 (8.892), and all academic SOTA models (CatVTON at 6.663 being the best of those). Academic models show a pronounced weakness in Garment Fidelity (e.g., CatVTON scores 4.007), which the authors attribute to insufficient robustness in complex scenarios. On multi-garment evaluation, Tstars-Tryon 1.0 reaches 9.171 versus 8.914 for Seedream5 lite, while FireRed-Image-Edit-1.1 collapses from 8.863 (single) to 4.822 (multi)—a performance collapse the authors attribute to garment omission, unresolved layering/occlusion, and catastrophic generative failures when visual conditions exceed model capacity. This contrast supports the claim that task-specific multi-reference training, rather than general editing capability alone, is what sustains performance under compositional stress.
On VITON-HD and DressCode under the unpaired setting—with no training data from either benchmark—the system attains FID/KID of 8.485/0.528 and 4.541/0.458 respectively, achieving state-of-the-art or competitive results against FastFit, FitDiT, CatVTON, and Leffa, evidencing zero-shot generalization to unseen distributions.
Human evaluation
Pairwise "Good/Same/Bad" comparisons against Nano Banana Pro and Seedream5 lite corroborate the quantitative findings. Against Nano Banana Pro, Tstars-Tryon 1.0 wins 41.1%, ties 41.6%, and loses 17.3%; against Seedream5 lite it wins 54.4% versus 9.0%. Notably, the paper observes a divergence between absolute metric scores and pairwise preference: Seedream5 lite slightly outscored Nano Banana Pro quantitatively yet lost more decisively in human evaluation, since GSB measures relative preference frequency rather than score magnitude. Win rates widen sharply with complexity—for example, rising from 46.1% (1 garment) to 70.2% (5 garments) against Seedream5 lite—which aligns with the multi-garment robustness claims.
Qualitative analysis identifies recurring failure modes in competitors that the proposed system avoids: identity degradation during full-body replacement, hallucinated backgrounds, lost hand poses, color/style confusion between garments, logo hallucination inconsistent with references, and item omission under up to 6 references. Demonstrations further extend beyond standard human photography to 3D avatars, anime characters, classical paintings, and even non-anthropomorphic subjects, which the authors interpret as evidence of learned, generalizable garment semantics rather than overfitting to human pose priors—an interpretation supported qualitatively but not backed by dedicated cross-domain quantitative evaluation.
Industrial deployment
The system is live on the Taobao App as "AI Try-On," covering the full consumer journey from product detail pages through portrait upload to single/multi-garment generation and outfit exploration. The paper reports several million users served and tens of millions of fulfilled requests, with planned expansion targeting tens of millions of daily requests. The authors frame this as resolving the long-standing trade-off between consumer-facing serving cost and generation quality, enabled primarily by the 5B-parameter DiT and distillation-based acceleration.
Limitations and open questions
The paper is candid about several constraints. Closed-source competitor latencies are estimated via API calls and may include network overhead, so the reported speedup figures should be read with caution. The benchmark's VLM-based scoring depends on the judging model's calibration, and although human evaluation partially validates the protocol, the correlation between VLM scores and human preference is not formally quantified. The demographic distribution skews heavily toward young female subjects (74.9% female, 85% youth), so robustness conclusions for other demographics rest on thinner coverage. Cross-domain capabilities (avatars, paintings, non-human subjects) are demonstrated only qualitatively. Finally, the promised public release of the benchmark is stated as a plan rather than an accomplished fact, leaving reproducibility of the headline comparisons contingent on that release.
Conclusion
Tstars-Tryon 1.0 demonstrates that a task-specialized, multi-reference MMDiT editor trained with a carefully staged data, SFT, and RL pipeline can outperform both academic try-on models and frontier proprietary editors on realistic, multi-garment benchmarks while running at commercially viable latency. Its principal contributions are the demonstration that general-purpose editors degrade sharply under multi-reference conditioning, the introduction of a privacy-preserving, in-the-wild benchmark aligned with human preferences, and validation at genuine production scale. The main open questions concern the reliability of VLM-as-judge scoring, coverage of underrepresented demographics, and quantitative characterization of cross-domain generalization.