FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On
Abstract: Given a person and a garment image, virtual try-on (VTO) aims to synthesize a realistic image of the person wearing the garment, while preserving their original pose and identity. Although recent VTO methods excel at visualizing garment appearance, they largely overlook a crucial aspect of the try-on experience: the accuracy of garment fit -- for example, depicting how an extra-large shirt looks on an extra-small person. A key obstacle is the absence of datasets that provide precise garment and body size information, particularly for "ill-fit" cases, where garments are significantly too large or too small. Consequently, current VTO methods default to generating well-fitted results regardless of the garment or person size. In this paper, we take the first steps towards solving this open problem. We introduce FIT (Fit-Inclusive Try-on), a large-scale VTO dataset comprising over 1.13M try-on image triplets accompanied by precise body and garment measurements. We overcome the challenges of data collection via a scalable synthetic strategy: (1) We programmatically generate 3D garments using GarmentCode and drape them via physics simulation to capture realistic garment fit. (2) We employ a novel re-texturing framework to transform synthetic renderings into photorealistic images while strictly preserving geometry. (3) We introduce person identity preservation into our re-texturing model to generate paired person images (same person, different garments) for supervised training. Finally, we leverage our FIT dataset to train a baseline fit-aware virtual try-on model. Our data and results set the new state-of-the-art for fit-aware virtual try-on, as well as offer a robust benchmark for future research. We will make all data and code publicly available on our project page: https://johannakarras.github.io/FIT.
Summary
- The paper presents a large-scale, fit-annotated dataset that fills the gap in current virtual try-on research by incorporating precise measurement data.
- It utilizes a synthetic pipeline with physics-based draping and diffusion-based modeling to control both garment appearance and fit with structured measurement conditioning.
- Empirical results demonstrate superior geometric consistency, robust garment size manipulation, and effective sim2real generalization compared to previous methods.
FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On
Motivation and Problem Formulation
Recent work on Virtual Try-On (VTO) image synthesis has emphasized garment appearance transfer while largely ignoring garment fit, a critical factor for realistic user experiences and commercial applications. Existing VTO models typically render well-fitted clothing regardless of actual body or garment size, primarily due to the absence of large-scale datasets containing precise measurement annotations and diverse fit scenarios, including ill-fits (e.g., oversized or undersized garments). This omission impedes both downstream applications and research on personalized, measurement-aware VTO. FIT ("Fit-Inclusive Try-on") directly addresses this foundational gap by proposing a dataset, benchmark, and baseline method focused on metric fit accuracy.
FIT Dataset: Design and Generation
Synthetic Pipeline for Fit Diversity
FIT comprises over 1.13M training and 1,000 test triplets of upper-body try-on images, each annotated with exact person and garment measurements and supporting paired images for robust supervised training. Unlike prior datasets (e.g., ViTON-HD, DressCode, DeepFashion3D), which are limited by catalog imagery and lack ill-fit diversity and reliable annotations, FIT leverages a synthetic pipeline based on GarmentCode, a domain-specific framework for generating and draping 3D sewing patterns onto parametric human models.
The pipeline involves the following steps:
- Procedural garment generation and pattern draping using physics simulation, with explicit cross-body draping and box-mesh realignment to enable realistic rendering of ill-fitting garments.
- Synthetic-to-photorealistic retexturing using a finetuned flow-based diffusion model (Flux.1-dev), conditioned on text prompts and composite normal maps, to narrow the sim2real domain gap and strictly preserve geometry.
- Procedurally generated paired person images (fixed pose and subject, swapped garments), supporting supervised mask-free VTO training.
Figure 1: FIT data generation pipeline, combining GarmentCode physics-based draping, compositional retexturing, and paired image synthesis.
Annotation Structure and Diversity
Each FIT triplet consists of a try-on image, the conditioning layflat garment image, a paired person image, and measurement vectors for both the person and the garment. Diverse fit scenarios are systematically covered, ranging from very tight (e.g., size XS garment on a size XL person) to very loose (e.g., size 3XL garment on a size XS person), with measurement distributions representing realistic, physically plausible combinations.
Baseline Fit-Aware VTO Model
Architecture and Conditioning
The proposed Fit-VTO model is a diffusion-based image generator initialized from Flux.1-dev, with LoRA finetuning and custom measurement conditioning. Input conditioning is highly structured: latent encodings of the paired person image and garment image are combined with Fourier-embedded person and garment measurements, replacing typical CLIP/T5 text conditioning. The conditioning mechanism enables both appearance and fit to be faithfully controlled and disentangled. Measurement vectors can specify bust, waist, length, sleeve length, etc., with explicit cross-attention throughout the diffusion backbone.
Figure 2: Flow-based Fit-VTO architecture integrating image and measurement conditioning.
Training and Evaluation Protocols
The FIT dataset is combined with a large collection of online fashion images (with null or coarse measurements) for pretraining, before fine-tuning on the FIT set with precise annotations. Masked L1 distance is adopted to assess non-garment identity preservation in paired image synthesis, and standard VTO metrics (SSIM, FID, LPIPS, KID) as well as fit-fidelity IoU are used to measure try-on image quality.
Empirical Analysis
Paired Image Consistency
Explicit conditioning on identity maps and ground-truth normals in FIT enables Fit-VTO to generate paired person images with markedly superior geometric and identity consistency compared to VLM-generated, inpainting-based, or standard VTO-based pseudo-pairing methods.
Figure 3: Qualitative comparison of paired image generation, illustrating Fit-VTO's advantages in pose and identity preservation and artifact suppression.
Garment Fit Control and Size Manipulation
Fit-VTO realistically modulates rendered garment fit along person and garment size axes, robustly handling extreme ill-fit scenarios and enabling independent control of garment attributes such as width, length, and sleeve. Controllability is substantiated by systematic garment size variation on a fixed subject and precise adaptation to individual measurement changes.
Figure 4: Garment fit visualization across a range of garment sizes on a fixed person size, highlighting independent size control.
Figure 5: Qualitative results for individual garment measurement manipulation: width, length, and sleeve length.
Sim2Real Generalization
Although FIT is fully synthetic, strong realism is achieved through the retexturing pipeline and in-context augmentation. Qualitative results demonstrate competitive performance on both synthetic test data and real-world try-on benchmarks (e.g., VITON-HD), including for resizing and adjustment scenarios where real annotation is infeasible.
Figure 6: Fit-VTO accurately adjusting garment fit on real-world person images by modifying garment measurements.
Quantitative Comparison and Ablations
Fit-VTO achieves state-of-the-art size-aware IoU and strong performance across SSIM, FID, and LPIPS relative to contemporaneous methods such as Any2AnyTryon, Nano Banana Pro, COTTON, and IDM-VTON. Ablation studies highlight the essential roles of (1) FIT supervision (for measurement grounding), (2) custom measurement encoder, and (3) the use of real imagery for generalization. Simple text encoders or models trained solely on real imagery or FIT synthetic data struggle with measurement fidelity or domain shift, respectively. The complete pipeline produces the best empirical balance.
Limitations and Future Directions
Current FIT iterations are focused on upper-body apparel, standardized front-facing views, and relatively simple garment structures as determined by GarmentCode's capabilities. The main technical limitations are (a) insufficient representation of graduated garment tightness (due to visual indistinguishability in simulation), and (b) residual correlation between measured attributes, making truly independent control challenging. The model's effectiveness is bounded by the realism and coverage of the garment/body generator.
Further development will prioritize extending the dataset and model framework to accommodate lower and full-body garments, expand pose and camera diversity, and introduce more structurally complex apparel types. Improvements in simulation engines may also enable better annotation and visual representation of nuanced fit phenomena such as stretch or compressibility.
Conclusion
FIT provides a substantial advancement for metric, fit-aware VTO by pairing annotational richness and domain coverage with a matched generative modeling baseline. Through a synthetic, physics-based pipeline and measurement-aware conditional modeling, FIT-VTO achieves robust generalization, appearance fidelity, and accurate fit visualization, laying the foundation for further research on fit-conditioned image synthesis and personalization benchmarks in virtual try-on. The dataset and model architecture introduce new evaluation standards for future VTO approaches.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
A simple explanation of “FIT: A Large-Scale Dataset for Fit-Aware Virtual Try-On”
1) What is this paper about?
This paper is about making virtual try-on tools smarter. A virtual try-on system takes a picture of a person and a picture of a piece of clothing, then shows what that person would look like wearing it. The big problem today: most systems only copy the clothing’s look (color, pattern) but not how it actually fits. For example, they often make every shirt look perfectly sized, even if it’s an XL on an XS body.
The authors built a huge new dataset and a new AI model so virtual try-on can show true fit—tight, perfect, or loose—based on real measurements of both the person and the garment.
2) What questions did the researchers ask?
Here are the main questions the paper tries to answer:
- How can we teach virtual try-on tools to show realistic fit, not just looks?
- Can we create a large dataset that includes exact body and clothing measurements (like height, chest, and sleeve length), including “ill-fit” cases (too big or too small)?
- Can an AI model use these measurements to control fit in the final image?
- How do we make computer-made (synthetic) examples look photorealistic and still keep the correct shape and size?
3) How did they do it? (Methods explained simply)
To solve this, the team built a pipeline—a series of steps—like dressing a 3D mannequin and turning it into a real-looking photo, all while keeping the correct size and shape.
Here’s the process in everyday terms:
- Making 3D clothes with exact sizes:
- Think of sewing patterns (the blueprints for clothes). The team used a tool called GarmentCode to program these patterns with exact measurements (like shirt width or sleeve length).
- They “draped” these clothes on 3D human bodies of different sizes, using physics so the fabric falls and wrinkles realistically—like gravity acting on cloth.
- Turning 3D renders into photorealistic pictures:
- The first images look like video game graphics. To make them look real, they used an image-generating AI (a diffusion model) that “repaints” the surface with realistic textures (skin, fabric, buttons) while keeping the shape unchanged.
- To lock in the shape, they used “normal maps,” which are like bump guides that tell the AI the exact 3D surface of the body and garment (where it bulges, folds, or is flat).
- Creating matching “paired” people photos:
- For training, they need two images of the same person in the same pose but wearing different tops. This is hard to collect in real life.
- Because their 3D pipeline is controllable, they can easily change the shirt while keeping the body and pose identical. Then an AI fills in the final details so both images look real and keep the same identity (same face, hair, background, and body shape).
- Adding a layflat garment image:
- They also generate a “layflat” picture (like store photos of a shirt laid on a table) to use as the clothing reference.
- Training a fit-aware model (Fit-VTO):
- They train a new try-on model, Fit-VTO, that takes:
- a person image,
- a layflat clothing image,
- and measurements of both the person and the garment (like height, chest, shirt width, sleeve length).
- The model is built on a powerful image generator and includes a special “measurement encoder” so the numbers (like 90 cm bust, XL shirt) directly control the fit in the output.
What they built overall:
- FIT dataset: 1.13 million try-on examples, including many sizes (XS–3XL) and lots of “too big” and “too small” situations.
- Triplets per example: a try-on image (target), a paired person image (same person, different top), and a layflat garment image—plus exact body and clothing measurements.
4) What did they find, and why is it important?
Main results:
- The FIT dataset makes it possible to train models that care about fit, not just appearance. It includes precise measurements and many ill-fitting cases that real online photos rarely show.
- Their Fit-VTO model shows noticeably better fit accuracy than current systems. It can keep the person’s identity and the clothing’s look, while making the shirt realistically tight or loose based on the numbers it’s given.
- The model can change the garment size while keeping the person the same, and it updates the image in a believable way (more wrinkles for tight fits, more drape for loose fits).
- Their method for creating paired images (same person, different top) preserves identity better than other AI editing methods, which often change faces, poses, or body shape by mistake.
Why this matters:
- Fit is what shoppers care about most. Showing the correct fit can reduce bad surprises, help people compare sizes (like S vs. M vs. L), and lead to fewer returns.
- For researchers, FIT is a new benchmark that encourages working on fit-aware try-on, not just texture copying.
5) What’s the impact, and what are the limits?
Implications:
- Better online shopping experiences: people can see how clothes would actually sit on their bodies—snug, just right, or oversized.
- More control: you can experiment with sizes and styles to get a certain look (like intentionally oversized).
- Stronger research base: the dataset and code help others build and test fit-aware systems.
Current limits and future work:
- Right now, the focus is on upper-body garments in mostly front-facing views and simpler poses.
- Very “tight vs. very-very tight” is hard to show differently because both can look skin-tight in pictures.
- They plan to expand to pants and dresses, more complex poses, and more camera angles.
In short, this paper shows how to teach AI to care about fit by giving it the right data and the right signals (exact measurements). That moves virtual try-on much closer to how clothes really look on different bodies and sizes.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a concise list of unresolved issues and concrete avenues for future research identified from the paper.
- Limited garment scope: the dataset and model focus on upper-body garments only; no support for pants, skirts, dresses, outerwear, or multi-layered outfits (e.g., jackets over shirts).
- Restricted viewpoints and poses: training images are standardized front-facing with casual poses; robustness to side/back views, extreme poses, and in-the-wild camera viewpoints is untested.
- Occlusions and real-world clutter: handling of heavy occlusions (e.g., crossed arms, hair covering shoulders, bags, scarves) and complex backgrounds remains unevaluated.
- Layering interactions: the pipeline simulates tops and bottoms separately and fixes bottoms across pairs; realistic multi-layer cloth interactions (e.g., shirt under jacket) are not modeled.
- Material/elasticity realism: physics draping does not vary cloth elasticity, friction, or anisotropy in a way that aligns with real fabrics; the retextured “fabric type” is only visual and not coupled to physical parameters.
- Tightness ambiguity: very tight fits saturate to skin-conforming appearance in simulation; dataset and model cannot represent degrees of tightness or contact pressure differences.
- Measurement coverage: only 7 inputs (4 body + 3 garment for tops); critical factors like shoulder width, neck/armhole/hem circumferences, sleeve/bicep girth, shoulder slope, and garment ease are not represented.
- Measurement disentanglement: the model exhibits correlated changes across measurements (e.g., width increases affecting length), indicating insufficient disentanglement and limited independent control.
- Measurement availability in practice: the system assumes known person and garment measurements; there is no integrated body/garment measurement estimation pipeline from images or recommended acquisition protocol.
- Handling missing or noisy measurements: training mixes samples with null measurements (-1) but does not analyze robustness to measurement noise or partial/missing inputs at inference.
- Metric for “fit” fidelity: IoU of garment masks is a coarse proxy; no evaluation of measurement-level errors (e.g., predicted vs. target bust/length), ease distribution, per-pixel strain/tension, or physical plausibility metrics.
- Human perception studies: no user studies assessing perceived fit accuracy, comfort, or realism; no A/B tests comparing different tightness levels or materials.
- Generalization to real-world images: qualitative results on VITON-HD are shown, but no systematic evaluation on diverse, in-the-wild datasets (varied lighting, backgrounds, cameras).
- Bias and fairness: distributional coverage and performance across body shapes, sizes, skin tones, genders, and age groups are not analyzed; potential demographic performance gaps are unexplored.
- Photorealism vs. geometry preservation: the retexturing module claims geometry preservation via normal maps, but there is no quantitative analysis of geometry drift (e.g., silhouette, keypoints) after retexturing.
- Normal map reliance: training uses normal maps from a single estimator; sensitivity to normal estimation errors and potential gains from richer geometric signals (depth, SMPL-X, UV/body landmarks) are untested.
- Simulation reliability: ill-fit drapes required custom box-mesh realignment; the rate of simulation failures, artifacts, and collision errors—and their impact on downstream training—is not quantified.
- Validation of physics realism: there is no calibration of simulated drapes against real-world draping (e.g., paired scans/photos), leaving uncertainty about physical plausibility in extreme fit cases.
- Layflat image consistency: layflat garments are generated by a VLM; consistency between layflat and the draped garment (shape details, prints, closures) is not validated and may introduce training noise.
- Paired-person generation validation: identity preservation is measured by masked L1 only; no robust identity metrics (e.g., face embeddings), pose keypoint consistency, or background consistency evaluations.
- Hallucinated details via prompts: buttons, seams, and pockets are inpainted from text; the risk of hallucination inconsistent with pattern geometry is not quantified.
- Dataset diversity limits: garment styles are bounded by GarmentCode templates; coverage of complex constructions (e.g., darts, pleats, structured collars, raglan/dolman sleeves) is limited.
- Temporal consistency: no support for video try-on or multi-frame temporal coherence; stability under small viewpoint/pose changes is unexplored.
- 3D consistency and multi-view: the approach is purely image-based; generating consistent multi-view or 3D-consistent try-on results remains open.
- Real-world deployment aspects: runtime, memory footprint, mobile feasibility, and latency under practical constraints are not discussed.
- Training-data mixing strategy: the impact of combining synthetic FIT data with web images lacking measurements on measurement grounding and overfitting is not deeply analyzed.
- Measurement encoder design: Fourier features + MLP replace text encoders; comparisons against alternative numeric conditioning (e.g., learned tokenizers, structured priors) and calibration/monotonicity constraints are missing.
- Robustness to accessories: effects of necklaces, belts, watches, or underlayers that change silhouette are not addressed.
- Evaluating out-of-distribution garments: performance on garments unseen in GarmentCode (e.g., asymmetric cuts, non-planar panels, ruched/stretch designs) is unknown.
- Ethical considerations: potential body image impacts, misuse (e.g., non-consensual try-on), and privacy concerns stemming from face/identity synthesis are not addressed.
- Data and licensing transparency: while release is promised, the exact licensing, allowed use-cases, and any restrictions on commercial/derivative use are unspecified.
Practical Applications
Practical Applications Derived from FIT and Fit-VTO
Below are actionable use cases grounded in the paper’s dataset (FIT), synthetic-to-real re-texturing pipeline, paired-image generation framework, and measurement-conditioned Fit-VTO model. Each item names target sectors, outlines potential tools/products/workflows, and notes assumptions/dependencies.
Immediate Applications
- Ecommerce retail: Fit-aware virtual try-on on product pages
- Sectors: Retail/fashion, software, marketing
- What: Let shoppers toggle sizes (e.g., XS–3XL) and see realistic tight/loose fit on their own image; provide visual justification for size recommendations
- Tools/products/workflows:
- Fit-VTO API/SDK embedded in web/mobile product pages
- Measurement ingestion workflow: retailer-provided garment measurements (length, width, sleeve, waist), or automated extraction from patterns/CAD/layflat
- Lightweight user measurement capture (self-reported, phone-based measurement estimation) and privacy-preserving image handling
- Assumptions/dependencies:
- Retailers supply accurate, structured garment measurements
- Users provide an image and basic measurements; adherence to privacy/compliance
- Current FIT scope: upper-body garments, front-facing views; strong performance in those regimes
- Size recommendation with visual explanations
- Sectors: Retail/fashion, data science/analytics
- What: Pair numeric size recommendations with side-by-side renders of different sizes on the shopper to reduce uncertainty and returns
- Tools/products/workflows:
- Fit-VTO as a post-processing visualizer for existing recommendation engines
- A/B testing and return-rate analytics pipeline
- Assumptions/dependencies:
- Quality of measurement estimation strongly affects trust; calibration per brand fit profile may be required
- Catalog and merchandising content generation across bodies and sizes
- Sectors: Retail/fashion, marketing/content operations
- What: Produce inclusive imagery showing fit on diverse bodies and sizes without multiple photoshoots
- Tools/products/workflows:
- Re-texturing pipeline (Flux-based) for photorealistic outputs preserving geometry
- Paired identity-preserving image generation to create consistent campaigns
- Brand style LoRA adapters for visual alignment
- Assumptions/dependencies:
- Usage rights for model fine-tuning and images; disclosure policies for AI-generated content
- QC processes for on-brand visuals; initial focus on upper-body products
- Peer-to-peer marketplaces and resale listings with fit visuals
- Sectors: Marketplaces, circular economy
- What: Sellers upload layflat images/measurements; buyers preview fit on themselves, reducing misfit-related returns
- Tools/products/workflows:
- Seller-side measurement capture wizard (guided entry)
- Buyer-side Fit-VTO widget integrated with listings
- Assumptions/dependencies:
- Consistent measurement practices by sellers; basic user measurement capture
- Designer prototyping for upper-body garments
- Sectors: Fashion design, product development
- What: Rapidly preview how pattern parameter changes affect fit across sizes and bodies before sampling
- Tools/products/workflows:
- GarmentCode-based procedural pattern adjustment + physics drape
- Re-texturing for client/stakeholder-ready visuals
- Assumptions/dependencies:
- Best suited to simpler upper-body structures; designers validate against physical samples
- Academic and industrial R&D on fit-aware generative models
- Sectors: Academia, ML/vision labs, applied research
- What: Benchmarking fit-aware VTO and measurement-conditioned diffusion; study sim-to-real with geometry constraints
- Tools/products/workflows:
- FIT dataset (1.13M triplets with measurements) for training and evaluation
- IoU-based size fidelity metric; paired-image generation framework
- Assumptions/dependencies:
- Adherence to dataset license; acknowledging FIT limitations (upper-body focus, standardized views)
- Accessibility and inclusivity in online shopping
- Sectors: Retail/fashion, DEI programs
- What: Show fit for a broader range of body shapes and sizes in product pages, enabling more confident purchases
- Tools/products/workflows:
- Curated body-shape presets and measurement sliders
- Assumptions/dependencies:
- Coverage is limited to body shapes represented in FIT; ongoing expansion is beneficial
- In-store clienteling and kiosks
- Sectors: Retail (brick-and-mortar), omnichannel
- What: Associate- or self-serve stations for quick body measurement capture and size-visualization across inventory
- Tools/products/workflows:
- Edge-deployable Fit-VTO inference or secure cloud endpoint; camera capture workflow with consent
- Assumptions/dependencies:
- Network or GPU availability; clear privacy and consent flows
Long-Term Applications
- Live, full-body AR try-on with motion and multi-garment layering
- Sectors: Retail/fashion, AR/VR, software
- What: Real-time, pose-diverse, multi-view, layered try-on with dynamic cloth behavior in mobile apps and smart mirrors
- Tools/products/workflows:
- Expanded datasets (lower-body, dresses, outerwear), multi-camera calibration, temporal diffusion/3D models, efficient on-device inference
- Assumptions/dependencies:
- Significant R&D for temporal consistency and fast cloth warping; broader dataset coverage beyond upper-body
- Made-to-measure and on-demand manufacturing
- Sectors: Apparel manufacturing, supply chain, customization
- What: Use user measurements and preferred fit to auto-adjust sewing patterns and visualize outcomes pre-production
- Tools/products/workflows:
- GarmentCode pattern parameterization connected to PLM/CAD; Fit-VTO as customer-facing preview; pipeline to cut/sew facilities
- Assumptions/dependencies:
- Robust measurement capture; brand- and fabric-specific grading rules; material properties catalog
- Standardized measurement schema and fit-accuracy certification
- Sectors: Policy/standards bodies, retail consortia
- What: Industry-wide schema for garment measurements and APIs; certification/audit programs for VTO fit claims
- Tools/products/workflows:
- Open schema (e.g., bust/waist/hips/length/sleeve with units and tolerances), brand mappings, third-party verification protocols
- Assumptions/dependencies:
- Multi-stakeholder coordination; consumer protection frameworks for AI marketing claims
- Inventory planning and sustainability analytics
- Sectors: Retail operations, ESG
- What: Simulate fit-driven purchase behavior and size demand; reduce overproduction and returns waste
- Tools/products/workflows:
- Demand forecasting models augmented with fit-visualization engagement; dashboards tying VTO usage to return-rate and carbon metrics
- Assumptions/dependencies:
- Reliable telemetry linking VTO exposure to outcomes; causal inference methods for impact attribution
- Cross-brand size harmonization and fit search
- Sectors: Retail, aggregators/marketplaces
- What: Translate brand size labels into common measurement primitives; let users search “fits like my L in Brand A” across brands
- Tools/products/workflows:
- Brand-specific measurement mappings; Fit-VTO previews to validate equivalence
- Assumptions/dependencies:
- High-quality brand measurement catalogs; handling style-specific ease allowances
- Virtual styling assistants and outfit fit-compatibility
- Sectors: Retail/fashion tech, personal styling
- What: Recommend multi-item outfits that fit together (e.g., jacket over hoodie) and visualize layering realism
- Tools/products/workflows:
- Expanded multi-garment simulation/training; collision-aware conditioning; measurement-aware diffusion prompts
- Assumptions/dependencies:
- Data for layer interactions and materials; advanced physics or learned priors
- Digital fashion and avatar ecosystems
- Sectors: Gaming/metaverse, digital goods
- What: Fit-aware apparel on avatars with accurate drape by measurement; sell size-responsible digital apparel
- Tools/products/workflows:
- Plugins for game engines; avatar-to-measurement mapping; Fit-VTO style generators adapted to 3D/real-time
- Assumptions/dependencies:
- Runtime constraints; integration with avatar pipelines; IP rights for digital garments
- Healthcare and wearables fit visualization
- Sectors: Medtech, sports/wearables
- What: Simulate fit of braces, supports, or smart wearables on bodies with varying anthropometrics
- Tools/products/workflows:
- Domain-adapted measurement-conditioned models; clinical validation; material-specific simulation
- Assumptions/dependencies:
- Safety/efficacy validation; specialized measurements and regulatory oversight
- Robotics and automated handling of garments (perception side)
- Sectors: Robotics, logistics
- What: Use synthetic draped imagery and measurement-conditioned generators to pretrain perception for garment recognition, grasp-point detection
- Tools/products/workflows:
- Sim2real pipelines bridging photorealistic cloth images to robot vision tasks
- Assumptions/dependencies:
- 3D annotations and physics expansion of the dataset; integration with robot control policies
- Longitudinal wellness and body-change visualization
- Sectors: Health/fitness apps
- What: Visualize how clothing fit evolves with body-size changes over time in a privacy-preserving app
- Tools/products/workflows:
- Periodic measurement capture; on-device generation; user consent and data minimization
- Assumptions/dependencies:
- Strong privacy safeguards; careful UX to avoid harm (e.g., body image concerns)
Key Cross-Cutting Assumptions and Dependencies
- Data scope and realism: Current FIT focuses on upper-body garments and standardized front-facing views; tightness gradations and complex, multi-layered apparel are limited. Generalization to dynamic scenes and lower-body garments requires further data and modeling.
- Measurement availability and quality: High-quality structured garment measurements and reliable user measurement capture are prerequisites for accurate fit visualization and recommendations.
- Brand/style variability: Brand-specific pattern grading and stylistic ease vary; harmonization layers are necessary for cross-brand comparability.
- Privacy, rights, and compliance: Using user images requires strong privacy practices; AI-generated imagery may need disclosure per jurisdiction; licensing constraints for base models/datasets apply.
- Compute and latency: Real-time or large-scale deployments need optimization (e.g., distillation, on-device acceleration, or efficient cloud inference).
- Fabric/material modeling: Realistic depiction of material behavior and multi-garment interactions will require expanded datasets and/or physics-informed models.
Glossary
- 3D grounding: Ensuring generated images are consistent with underlying 3D geometry and segmentation. "to produce photorealistic try-on triplets with 3D grounding."
- box-mesh realignment: Realigning initial garment pattern meshes to the target body before draping to avoid simulation failures. "using box-mesh realignment to prevent simulation failures."
- channel-concatenated: Concatenation along the channel dimension of latent tensors for conditioning. "Person latents are channel-concatenated with the noisy target latents"
- CLIP: A contrastive language–image pretraining model used for text/image conditioning; here removed to use measurement embeddings instead. "We remove the CLIP and T5 text conditionings for Flux.1-dev"
- composite normal map: A surface normal map composed or stitched from multiple sources to better preserve geometry. "and a composite normal map based on ."
- cross-attention: A transformer mechanism that conditions generation by attending from one sequence (image tokens) to another (measurement embeddings). "via cross-attention, replacing the T5 text conditioning in the single-stream and double-stream blocks."
- cross-drape: Draping a garment pattern designed for one body onto a different target body to evaluate fit. "Then, we cross-drape the pattern onto a different target body of size "
- diffusion model: A generative model that iteratively denoises noise into an image, conditioned on guidance signals. "we fine-tune a diffusion model, $f_{\text{texture}$ (based on Flux.1-dev)"
- domain gap: The discrepancy between synthetic and real image distributions that can hinder generalization. "a significant domain gap persists between real-world and synthetic data."
- domain-specific language: A specialized programming language for a particular domain; here, for specifying sewing patterns with size parameters. "introducing a domain-specific language for generating sewing patterns with explicit size parameters"
- FID: Fréchet Inception Distance, a metric for measuring the quality of generated images relative to real images. "We compute common VTO metrics -- SSIM~\cite{ssim}, FID~\cite{fid}, LPIPS~\cite{lpips}, KID~\cite{kid}"
- FLUX-Controlnet-Inpainting: A specific inpainting model variant combining FLUX with ControlNet conditioning. "FLUX-Controlnet-Inpainting \cite{fluxinpaint}"
- flow-matching diffusion model: A diffusion formulation that predicts velocity fields to match a rectified flow between noise and data. "Our architecture (Figure~\ref{fig:architecture}) is a flow-matching diffusion model"
- Flux.1-dev: A large pre-trained text-to-image diffusion/flow model used as the base for fine-tuning. "We replace the text embeddings in Flux.1-dev with custom measurement embeddings"
- Fourier Feature Embeddings: Sinusoidal encodings that map continuous numeric inputs (e.g., measurements) into higher-dimensional features. "we compute the Fourier Feature Embeddings for each measurement"
- guidance scale: The strength of conditioning (e.g., classifier-free guidance) during inference for controlling adherence to inputs. "At inference, we set guidance scale to 1.0"
- identity map: A mask preserving non-garment regions (e.g., background/skin) from the original image to maintain identity during editing. "we derive an identity map by masking out the combined source and paired garment regions"
- identity preservation: Maintaining the same person’s appearance (face/body) across generated images. "we introduce person identity preservation into our re-texturing model"
- inpainting: Filling in masked image regions using generative models, conditioned on context. "we employ a conditional inpainting model $f_{\text{paired}$"
- Intersection-Over-Union (IoU): A metric measuring overlap between predicted and ground-truth masks. "IoU measures the Intersection-Over-Union of the garment mask in synthesized try-on image and the ground truth."
- KID: Kernel Inception Distance, a generative image quality metric similar to FID but unbiased. "We compute common VTO metrics -- SSIM~\cite{ssim}, FID~\cite{fid}, LPIPS~\cite{lpips}, KID~\cite{kid}"
- layflat garment image: An image of a garment laid flat (not worn) used as conditioning input for try-on. "to generate a layflat garment image "
- LoRA: Low-Rank Adaptation, a parameter-efficient fine-tuning technique that adds small low-rank layers to large models. "We finetune only the lightweight LoRA parameters"
- LPIPS: Learned Perceptual Image Patch Similarity, a perceptual similarity metric between images. "We compute common VTO metrics -- SSIM~\cite{ssim}, FID~\cite{fid}, LPIPS~\cite{lpips}, KID~\cite{kid}"
- mask-based: Try-on approaches that rely on explicit segmentation masks to localize editing/generation. "a mask-based ``teacher'' model"
- mask-free: Try-on approaches that operate without segmentation masks, synthesizing results end-to-end. "Another line of research \cite{issenhuth2020not,ge2021disentangled,ge2021parser,dumitigating,du2023greatness,zhang2025boow,any2anytryon} focus on mask-free architectures."
- measurement embeddings: Learned vector representations of numerical person/garment measurements used to condition generation. "with custom measurement embeddings $m_{\text{embed}$ computed from ."
- measurement encoder: A module that maps raw measurement vectors into embeddings for cross-attention conditioning. "from our custom measurement encoder ."
- Multi-modal Diffusion Transformer (MMDiT): A transformer backbone architecture for diffusion models that handles multiple modalities. "employs a rectified flow formulation and a Multi-modal Diffusion Transformer (MMDiT) backbone"
- normal map: An image encoding per-pixel surface normals used to preserve geometry across domains. "conditioned on an input normal map and a text prompt ."
- parametric body model: A body representation controlled by parameters (e.g., shape/size) for systematic variation. "from GarmentCode's parametric body model"
- physics simulation: Computational simulation of physical cloth dynamics to realistically drape garments. "drape them via physics simulation to capture realistic garment fit."
- Prodigy optimizer: A specific optimization algorithm used to train the model. "We adopt Prodigy optimizer with learning rate $1.0$"
- pseudo-triplets: Synthetic training triplets (person, garment, target) generated without real paired data. "these methods typically rely on generating ``pseudo-triplets'' via generative modeling"
- rectified flow: A training/inference formulation for flow-based diffusion that straightens the flow between noise and data. "employs a rectified flow formulation and a Multi-modal Diffusion Transformer (MMDiT) backbone"
- re-texturing: Transforming synthetic renderings into photorealistic images while preserving geometry. "we employ a novel re-texturing pipeline"
- sequence-wise concatenated: Concatenation along the token/sequence dimension to merge conditioning streams. "layflat latents and $m_{\text{embed}$ are sequence-wise concatenated with ."
- size leakage: Unintended transfer of size information from source to target during training, corrupting conditioning. "which suffer from inaccurate masking, identity loss, and size leakage."
- SSIM: Structural Similarity Index Measure, an image similarity metric assessing perceived quality. "We compute common VTO metrics -- SSIM~\cite{ssim}, FID~\cite{fid}, LPIPS~\cite{lpips}, KID~\cite{kid}"
- T5: A pre-trained text encoder (Text-to-Text Transfer Transformer) used in some diffusion models for textual conditioning. "We remove the CLIP and T5 text conditionings for Flux.1-dev"
- VAE: Variational Autoencoder, used here for encoding/decoding image latents in the diffusion pipeline. "pre-trained VAE encoder"
- VLM: Vision-LLM used to generate or interpret text prompts from images. "we generate a text prompt (via VLM)"
- virtual try-on (VTO): Synthesizing images of a person wearing a specified garment while preserving their identity and pose. "virtual try-on (VTO) aims to synthesize a realistic image of the person wearing the garment"
- warp-based simulation algorithm: A cloth simulation approach (here via NVIDIA Warp) used to drape garments. "using a warp-based simulation algorithm~\cite{macklin2022warp}"
- warping-based works: Early try-on methods that warp garment images to align with person images before refinement. "Early warping-based works~\cite{viton, viton-hd} established a two-stage paradigm"
Collections
Sign up for free to add this paper to one or more collections.