SwiftTailor: Efficient 3D Garment Generation

• SwiftTailor is a two-stage framework that combines sewing-pattern prediction with mesh synthesis to generate realistic 3D garments efficiently.
• It employs a novel geometry image representation that encodes garment panels, semantic labels, and stitch topology in a unified UV-space layout for clear interpretability.
• SwiftTailor significantly reduces inference time compared to traditional physics-based methods while enabling scalable, simulation-ready garment production.

SwiftTailor is a two-stage framework for efficient, physically plausible, and interpretable 3D garment generation that introduces a novel geometry image representation to unify sewing-pattern prediction and mesh synthesis. Distinct from end-to-end deep generative and diffusion-based approaches that frequently neglect garment structure or rely on slow, resource-intensive physics simulators, SwiftTailor achieves state-of-the-art accuracy and visual fidelity while significantly reducing inference time, enabling scalable and practical applications in computer-aided design, virtual try-on, and digital fashion (Pham et al., 19 Mar 2026).

1. Problem Formulation and Principal Contributions

SwiftTailor addresses the longstanding challenge of realistic 3D garment generation that meets industrial requirements for interpretability, manufacturability, and simulation-readiness. Standard methods either discard the underlying sewing-pattern logic or require costly iterative physics simulation to assemble 3D meshes from predicted patterns (with runtimes of 30–60 seconds per garment). SwiftTailor revives the traditional two-stage workflow—explicit 2D sewing pattern inference, followed by 3D garment assembly—while amortizing the cost of sewing simulation through a unified geometry image representation and algorithmically efficient mesh construction.

Key technical contributions are:

• PatternMaker: An efficient, multimodal vision-LLM for sewing-pattern prediction from images, text descriptions, or both.
• Garment Geometry Image (GGI): A compact, UV-space image encoding all garment panels’ 3D geometry, semantic labels, and stitch topology.
• GarmentSewer: A dense prediction transformer that directly maps pattern semantics to a geometry image, bypassing iterative mesh simulation.
• Fast Inverse Mapping: A remeshing and dynamic stitching procedure that reconstructs a watertight, simulatable mesh within ≈0.02 s, eliminating the need for physics-based solvers.

2. PatternMaker: Architecture and Training

PatternMaker builds on the InternVL-3-2B backbone (≈2B parameters), which is fine-tuned with AIpparel’s sewing-pattern tokenization and trained to handle diverse input modalities: single or paired garment images, text descriptions, or multimodal prompts. The predicted sewing pattern $\mathcal P$ consists of a set of panels $P_i=(V_i,E_i,R_i)$—planar vertices $V_i\subset\mathbb R^2$, edges $E_i$, and a rigid transform $R_i\in SE(3)$—along with stitch tuples $s_k=(e_a,e_b)$ pairing boundary edges.

The training objective is multi-headed:

• Discrete sewing-pattern token prediction via cross-entropy:

$$\mathcal L_{\rm tok} = -\sum_{t}\log p(y_t\mid y_{<t},x)$$

• Continuous regression heads for vertices, rotations (Frobenius norm), and translations:

$$\mathcal L_{\rm vert} = \sum_{i,j}\|\hat v_{ij}-v_{ij}\|_2^2, \quad \mathcal L_{\rm rot} = \|\hat R_i - R_i\|_F^2, \quad \mathcal L_{\rm trans} = \|\hat t_i - t_i\|_2^2$$

PatternMaker delivers superior accuracy on Vertex L2, panel-count, rotation, and stitch-matching benchmarks, outperforming LLaVA-7B and prior models such as AIpparel, ChatGarment, and SewingLDM (Pham et al., 19 Mar 2026).

3. Garment Geometry Image and GarmentSewer

The Garment Geometry Image (GGI) encodes the 3D surface for all panels in a shared UV-space layout $L_{\rm UV} \subset [0,1]^2$. It comprises three aligned $H \times W$ maps:

• Semantic map $P_i=(V_i,E_i,R_i)$0: Encodes panel type as color labels.
• Geometry map $P_i=(V_i,E_i,R_i)$1: Densely parameterizes the surface at each UV coordinate.
• Stitch map $P_i=(V_i,E_i,R_i)$2: Indexes of edges to be sewn.

Geometry-image construction proceeds by rasterizing mesh vertices in UV space, interpolating geometry along panel edges (linear) and interiors (barycentric), ensuring bijective mapping for each panel $P_i=(V_i,E_i,R_i)$3.

GarmentSewer, a dense prediction transformer (ViT-L encoder + U-Net style decoder), generates a dense geometry map $P_i=(V_i,E_i,R_i)$4 conditioned on $P_i=(V_i,E_i,R_i)$5 via:

• Edge-aware L1 regression:

$P_i=(V_i,E_i,R_i)$6

• Stitch alignment via Chamfer loss:

$P_i=(V_i,E_i,R_i)$7

• Normal-smoothness regularization:

$P_i=(V_i,E_i,R_i)$8

The total objective is a weighted sum of these components.

This module delivers semantic-to-geometry mapping efficiently and is modular for integration with other garment pipelines.

4. Mesh Reconstruction: Remeshing and Dynamic Stitching

Once a GGI is predicted, mesh reconstruction is realized in two substeps:

1. UV-aligned remeshing: Each $P_i=(V_i,E_i,R_i)$9 UV cell is checked for valid 3D points. If three are valid, a single triangle is created; for all four, the cell is split along the shorter diagonal.
2. Dynamic Stitching: The predicted stitch map $V_i\subset\mathbb R^2$0 yields boundary curves; Dynamic Time Warping (DTW) aligns corresponding panel boundaries, and a Disjoint-Set-Union (DSU) merges matched vertex indices, averaging positions and removing degenerate faces. The result is a single watertight mesh matching the intended sewing pattern, with assembly performed in ≈0.02 s—orders of magnitude faster than traditional physics-based solvers (Pham et al., 19 Mar 2026).

5. Experimental Evaluation

Experiments were conducted on GCD-MM (multimodal GarmentCodeData extension) using standard train/val/test splits on 4 A100 GPUs for three days. Evaluation metrics included:

• Pattern quality: Vertex L2, Panel-count Acc, Edge Acc, Rotation L2, Translation L2, Stitch Acc
• Mesh quality: Minimum Matching Distance (MMD↓), Coverage (COV↑), sampling attempts, inference time

Quantitative results:

Method	MMD	COV	Inference Time
SwiftTailor	5.31	0.68	14.78 s
PatternMaker+GC	6.82	0.54	49–63 s

Qualitatively, SwiftTailor produces cleaner seams, no tearing, and consistent draping across input modalities. Inference time for mesh assembly is reduced by approximately 4× compared to GarmentCode-based pipelines (Pham et al., 19 Mar 2026).

6. Applications, Limitations, and Future Directions

SwiftTailor’s modularity allows PatternMaker and GarmentSewer to be integrated independently into other digital fashion or CAD systems. The explicit, interpretable representations enable downstream tasks such as texture mapping, mesh editing, and CAD integration. The approach is notably scalable and eliminates the need for expensive physics simulation at deployment.

Known limitations include reduced fidelity of high-frequency wrinkles and sensitivity to out-of-distribution inputs (occlusions, rare silhouettes). Possible extensions include real-time pattern generation, GGI-conditioned texture/material synthesis, and refined wrinkle modeling via learned or physics-guided approaches.

A plausible implication is that SwiftTailor’s efficiency and modularity set a new benchmark for scalable, interpretable 3D garment generation in both academic and industrial settings (Pham et al., 19 Mar 2026).