Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Significance of Anatomical Constraints in Virtual Try-On (2401.02110v1)

Published 4 Jan 2024 in cs.CV

Abstract: The system of Virtual Try-ON (VTON) allows a user to try a product virtually. In general, a VTON system takes a clothing source and a person's image to predict the try-on output of the person in the given clothing. Although existing methods perform well for simple poses, in case of bent or crossed arms posture or when there is a significant difference between the alignment of the source clothing and the pose of the target person, these methods fail by generating inaccurate clothing deformations. In the VTON methods that employ Thin Plate Spline (TPS) based clothing transformations, this mainly occurs for two reasons - (1)~the second-order smoothness constraint of TPS that restricts the bending of the object plane. (2)~Overlaps among different clothing parts (e.g., sleeves and torso) can not be modeled by a single TPS transformation, as it assumes the clothing as a single planar object; therefore, disregards the independence of movement of different clothing parts. To this end, we make two major contributions. Concerning the bending limitations of TPS, we propose a human AnaTomy-Aware Geometric (ATAG) transformation. Regarding the overlap issue, we propose a part-based warping approach that divides the clothing into independently warpable parts to warp them separately and later combine them. Extensive analysis shows the efficacy of this approach.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. B. Wang, H. Zheng, X. Liang, Y. Chen, L. Lin, and M. Yang, “Toward characteristic-preserving image-based virtual try-on network,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 589–604.
  2. R. Yu, X. Wang, and X. Xie, “Vtnfp: An image-based virtual try-on network with body and clothing feature preservation,” in Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 10 511–10 520.
  3. D. Roy, S. Santra, and B. Chanda, “Lgvton: a landmark guided approach for model to person virtual try-on,” Multimedia Tools and Applications, pp. 1–37, 2022.
  4. ——, “Incorporating human body shape guidance for cloth warping in model to person virtual try-on problems,” 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), pp. 1–6, 2020.
  5. X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7794–7803.
  6. H. Yang, R. Zhang, X. Guo, W. Liu, W. Zuo, and P. Luo, “Towards photo-realistic virtual try-on by adaptively generating-preserving image content,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  7. H. Dong, X. Liang, X. Shen, B. Wang, H. Lai, J. Zhu, Z. Hu, and J. Yin, “Towards multi-pose guided virtual try-on network,” in Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 9026–9035.
  8. A. Chopra, R. Jain, M. Hemani, and B. Krishnamurthy, “Zflow: Gated appearance flow-based virtual try-on with 3d priors,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5433–5442.
  9. Y. Ge, Y. Song, R. Zhang, C. Ge, W. Liu, and P. Luo, “Parser-free virtual try-on via distilling appearance flows,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8485–8493.
  10. X. Han, X. Hu, W. Huang, and M. R. Scott, “Clothflow: A flow-based model for clothed person generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 10 471–10 480.
  11. S. He, Y.-Z. Song, and T. Xiang, “Style-based global appearance flow for virtual try-on,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3470–3479.
  12. T. Zhou, S. Tulsiani, W. Sun, J. Malik, and A. A. Efros, “View synthesis by appearance flow,” in European conference on computer vision.   Springer, 2016, pp. 286–301.
  13. X. Han, Z. Wu, Z. Wu, R. Yu, and L. S. Davis, “Viton: An image-based virtual try-on network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7543–7552.
  14. Z. Xie, Z. Huang, F. Zhao, H. Dong, M. Kampffmeyer, and X. Liang, “Towards scalable unpaired virtual try-on via patch-routed spatially-adaptive gan,” Advances in Neural Information Processing Systems, vol. 34, pp. 2598–2610, 2021.
  15. Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  16. T. Beier and S. Neely, “Feature-based image metamorphosis,” ACM SIGGRAPH computer graphics, vol. 26, no. 2, pp. 35–42, 1992.
  17. C.-W. Hsieh, C.-Y. Chen, C.-L. Chou, H.-H. Shuai, and W.-H. Cheng, “Fit-me: Image-based virtual try-on with arbitrary poses,” in 2019 IEEE International Conference on Image Processing (ICIP).   IEEE, 2019, pp. 4694–4698.
  18. N. Zheng, X. Song, Z. Chen, L. Hu, D. Cao, and L. Nie, “Virtually trying on new clothing with arbitrary poses,” in Proceedings of the 27th ACM International Conference on Multimedia.   ACM, 2019, pp. 266–274.
  19. C.-W. Hsieh, C.-Y. Chen, C.-L. Chou, H.-H. Shuai, J. Liu, and W.-H. Cheng, “Fashionon: Semantic-guided image-based virtual try-on with detailed human and clothing information,” in Proceedings of the 27th ACM International Conference on Multimedia.   ACM, 2019, pp. 275–283.
  20. F. Sun, J. Guo, Z. Su, and C. Gao, “Image-based virtual try-on network with structural coherence,” in 2019 IEEE International Conference on Image Processing (ICIP).   IEEE, 2019, pp. 519–523.
  21. Z. Wu, G. Lin, Q. Tao, and J. Cai, “M2e-try on net: Fashion from model to everyone,” in Proceedings of the 27th ACM International Conference on Multimedia.   ACM, 2019, pp. 293–301.
  22. H. Dong, X. Liang, X. Shen, B. Wu, B.-C. Chen, and J. Yin, “Fw-gan: Flow-navigated warping gan for video virtual try-on,” in Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 1161–1170.
  23. A. Neuberger, E. Borenstein, B. Hilleli, E. Oks, and S. Alpert, “Image based virtual try-on network from unpaired data,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  24. S. Jandial, A. Chopra, K. Ayush, M. Hemani, B. Krishnamurthy, and A. Halwai, “Sievenet: A unified framework for robust image-based virtual try-on,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2020, pp. 2182–2190.
  25. M. Zanfir, A.-I. Popa, A. Zanfir, and C. Sminchisescu, “Human appearance transfer,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  26. G. Yildirim, N. Jetchev, R. Vollgraf, and U. Bergmann, “Generating high-resolution fashion model images wearing custom outfits,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019, pp. 0–0.
  27. Y. Men, Y. Mao, Y. Jiang, W.-Y. Ma, and Z. Lian, “Controllable person image synthesis with attribute-decomposed gan,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 5084–5093.
  28. K. M. Lewis, S. Varadharajan, and I. Kemelmacher-Shlizerman, “Tryongan: body-aware try-on via layered interpolation,” ACM Transactions on Graphics (TOG), vol. 40, no. 4, pp. 1–10, 2021.
  29. A. H. Raffiee and M. Sollami, “Garmentgan: Photo-realistic adversarial fashion transfer,” in 2020 25th International Conference on Pattern Recognition (ICPR).   IEEE, 2021, pp. 3923–3930.
  30. S. Choi, S. Park, M. Lee, and J. Choo, “Viton-hd: High-resolution virtual try-on via misalignment-aware normalization,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14 131–14 140.
  31. R. Alp Güler, N. Neverova, and I. Kokkinos, “Densepose: Dense human pose estimation in the wild,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7297–7306.
  32. W. Liu, Z. Piao, J. Min, W. Luo, L. Ma, and S. Gao, “Liquid warping gan: A unified framework for human motion imitation, appearance transfer and novel view synthesis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5904–5913.
  33. S. Bai, H. Zhou, Z. Li, C. Zhou, and H. Yang, “Single stage virtual try-on via deformable attention flows,” in European Conference on Computer Vision.   Springer, 2022, pp. 409–425.
  34. T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4401–4410.
  35. Z. Xie, Z. Huang, X. Dong, F. Zhao, H. Dong, X. Zhang, F. Zhu, and X. Liang, “Gp-vton: Towards general purpose virtual try-on via collaborative local-flow global-parsing learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23 550–23 559.
  36. F. Malagelada, M. Dalmau-Pastor, J. Vega, and P. Golano, “Elbow anatomy,” Sports injuries: prevention, diagnosis, treatment and rehabilitation, vol. 2, pp. 527–53, 2014.
  37. V. Masteikaitě, V. Sacevičienė, and V. Čironienė, “Compressed loop method for the bending behaviour of coated and laminated fabrics analysis,” Journal of Industrial Textiles, vol. 43, no. 3, pp. 350–365, 2014.
  38. K. Gong, X. Liang, D. Zhang, X. Shen, and L. Lin, “Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 932–940.
  39. K. Lin, L. Wang, K. Luo, Y. Chen, Z. Liu, and M.-T. Sun, “Cross-domain complementary learning using pose for multi-person part segmentation,” IEEE Transactions on Circuits and Systems for Video Technology, 2020.
  40. G. Liu, F. A. Reda, K. J. Shih, T.-C. Wang, A. Tao, and B. Catanzaro, “Image inpainting for irregular holes using partial convolutions,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 85–100.
  41. B. Fele, A. Lampe, P. Peer, and V. Struc, “C-vton: Context-driven image-based virtual try-on network,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2022, pp. 3144–3153.
  42. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” in Advances in Neural Information Processing Systems, 2017, pp. 6626–6637.
  43. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli et al., “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  44. Z. Liu, P. Luo, S. Qiu, X. Wang, and X. Tang, “Deepfashion: Powering robust clothes recognition and retrieval with rich annotations,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1096–1104.
  45. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
  46. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09, 2009.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Debapriya Roy (5 papers)
  2. Sanchayan Santra (8 papers)
  3. Diganta Mukherjee (21 papers)
  4. Bhabatosh Chanda (11 papers)
Citations (1)
View on Semantic Scholar