Papers
Topics
Authors
Recent
Search
2000 character limit reached

Bayesian Differentiable Physics for Cloth Digitalization

Published 27 Feb 2024 in cs.CV | (2402.17664v4)

Abstract: We propose a new method for cloth digitalization. Deviating from existing methods which learn from data captured under relatively casual settings, we propose to learn from data captured in strictly tested measuring protocols, and find plausible physical parameters of the cloths. However, such data is currently absent, so we first propose a new dataset with accurate cloth measurements. Further, the data size is considerably smaller than the ones in current deep learning, due to the nature of the data capture process. To learn from small data, we propose a new Bayesian differentiable cloth model to estimate the complex material heterogeneity of real cloths. It can provide highly accurate digitalization from very limited data samples. Through exhaustive evaluation and comparison, we show our method is accurate in cloth digitalization, efficient in learning from limited data samples, and general in capturing material variations. Code and data are available https://github.com/realcrane/Bayesian-Differentiable-Physics-for-Cloth-Digitalization

Authors (3)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (83)
  1. Bs en iso 9073-9:2008: Textiles. test methods for nonwovens: Determination of drapability including drape coefficient, 2008.
  2. Botorch: a framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33:21524–21538, 2020.
  3. Large steps in cloth simulation. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pages 43–54, 1998.
  4. An overview on deep learning-based approximation methods for partial differential equations. arXiv preprint arXiv:2012.12348, 2020.
  5. Pbns: physically based neural simulator for unsupervised garment pose space deformation. arXiv preprint arXiv:2012.11310, 2020.
  6. Neural cloth simulation. ACM Transactions on Graphics (TOG), 41(6):1–14, 2022.
  7. Estimating cloth simulation parameters from video. 2003.
  8. Dennis P Bishop. Fabrics: sensory and mechanical properties. Textile Progress, 26(3):1–62, 1996.
  9. Weight uncertainty in neural network. In International conference on machine learning, pages 1613–1622. PMLR, 2015.
  10. Estimating the material properties of fabric from video. In Proceedings of the IEEE international conference on computer vision, pages 1984–1991, 2013.
  11. Simulation of clothing with folds and wrinkles. In ACM SIGGRAPH 2005 Courses, pages 3–es. 2005.
  12. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010.
  13. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174, 2016.
  14. Learning to predict 3d objects with an interpolation-based differentiable renderer. Advances in Neural Information Processing Systems, 32, 2019.
  15. Research problems in clothing simulation. Computer-aided design, 37(6):585–592, 2005.
  16. Mechanics of elastic performance of textile materials: Part v: a study of the factors affecting the drape of fabrics—the development of a drape meter. Textile Research Journal, 20(8):539–548, 1950.
  17. Modeling and data-driven parameter estimation for woven fabrics. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 1–11, 2017.
  18. Billie J Collier. Measurement of fabric drape and its relation to fabric mechanical properties and subjective evaluation. Clothing and Textiles Research Journal, 10(1):46–52, 1991.
  19. Effects of interfacing type on shear and drape behavior of apparel fabrics. Clothing and Textiles Research Journal, 7(3):51–56, 1989.
  20. GE Cusick. 30—the resistance of fabrics to shearing forces: A study of the experimental method due to mörner and eeg-olofsson. Journal of the Textile Institute Transactions, 52(9):T395–T406, 1961.
  21. GE Cusick. 46—the dependence of fabric drape on bending and shear stiffness. Journal of the Textile Institute Transactions, 56(11):T596–T606, 1965.
  22. GE Cusick. 21—the measurement of fabric drape. Journal of the Textile Institute, 59(6):253–260, 1968.
  23. Mathematical and numerical simulation of geometry and mechanical behavior of sandwich composites reinforced with 1×\times× 1-rib-gaiting weft-knitted spacer fabric; compressional behavior. Composite Structures, 268:113952, 2021.
  24. High-dimensional bayesian optimization with sparse axis-aligned subspaces. In Uncertainty in Artificial Intelligence, pages 493–503. PMLR, 2021.
  25. Peter I Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018.
  26. A review of multi-scale numerical modeling of three-dimensional woven fabric. Composite Structures, 263:113685, 2021.
  27. Salvatore Gazzo. Characterisation of the mechanical behaviour of networks and woven fabrics with a discrete homogenization model. PhD thesis, Université de Lyon; Università degli studi (Catane, Italie), 2019.
  28. Fine-grained differentiable physics: a yarn-level model for fabrics. In International Conference on Learning Representations, 2022.
  29. Eigen v3. http://eigen.tuxfamily.org, 2010.
  30. Garnet++: Improving fast and accurate static 3d cloth draping by curvature loss. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1):181–195, 2020.
  31. Differentiable dynamics for articulated 3d human motion reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13190–13200, 2022.
  32. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  33. F. Hecht. New development in freefem++. J. Numer. Math., 20(3-4):251–265, 2012.
  34. Stochastic variational inference. Journal of Machine Learning Research, 2013.
  35. phiflow: A differentiable pde solving framework for deep learning via physical simulations. In NeurIPS Workshop, 2020.
  36. Kaolin: A pytorch library for accelerating 3d deep learning research. arXiv preprint arXiv:1911.05063, 2019.
  37. gradsim: Differentiable simulation for system identification and visuomotor control. arXiv preprint arXiv:2104.02646, 2021.
  38. YJ Jeong. A study of fabric-drape behaviour with image analysis part i: Measurement, characterisation, and instability. Journal of the Textile Institute, 89(1):59–69, 1998.
  39. A study of fabric-drape behaviour with image analysis. part ii: the effects of fabric structure and mechanical properties on fabric drape. Journal of the Textile Institute, 89(1):70–79, 1998.
  40. Estimating cloth simulation parameters from a static drape using neural networks. IEEE Access, 8:195113–195121, 2020.
  41. Physics-informed machine learning. Nat Rev Phys, (3):422–440, 2021.
  42. Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057, 2020.
  43. Near-exhaustive precomputation of secondary cloth effects. ACM Transactions on Graphics (TOG), 32(4):1–8, 2013.
  44. Sungmin Kim. Determination of fabric physical properties for the simulation of cusick drapemeter. Fibers and Polymers, 12(1):132–136, 2011.
  45. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  46. Re-examining linear embeddings for high-dimensional bayesian optimization. Advances in neural information processing systems, 33:1546–1558, 2020.
  47. Diffcloth: Differentiable cloth simulation with dry frictional contact. arXiv preprint arXiv:2106.05306, 2021.
  48. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Trans. Graph., 42(1), 2022a.
  49. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Transactions on Graphics (TOG), 42(1):1–20, 2022b.
  50. N-Cloth: Predicting 3D cloth deformation with mesh-based networks. Computer Graphics Forum (Proceedings of Eurographics), 41(2):547–558, 2022c.
  51. Differentiable cloth simulation for inverse problems. Advances in Neural Information Processing Systems, 32, 2019.
  52. Soft rasterizer: A differentiable renderer for image-based 3d reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7708–7717, 2019.
  53. Deepxde: A deep learning library for solving differential equations. SIAM Review, 63(1):208–228, 2021.
  54. The simulation of cloth using accurate physical parameters. CGIM 2008, Insbruck, Austria, 2008.
  55. When physics meets machine learning: A survey of physics-informed machine learning. arXiv preprint arXiv:2203.16797, 2022.
  56. Modeling and estimation of internal friction in cloth. ACM Transactions on Graphics (TOG), 32(6):1–10, 2013.
  57. Adaptive anisotropic remeshing for cloth simulation. ACM transactions on graphics (TOG), 31(6):1–10, 2012.
  58. A framework for bayesian optimization in embedded subspaces. In International Conference on Machine Learning, pages 4752–4761. PMLR, 2019.
  59. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  60. Tailornet: Predicting clothing in 3d as a function of human pose, shape and garment style. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7365–7375, 2020.
  61. P Pratihar. Static and dynamics drape of fabric: An emerging arena of fabric evaluation. International Journal of Engineering Research and Applications, 3(5):1007–1011, 2013.
  62. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  63. Cloth in the wind: A case study of physical measurement through simulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10498–10507, 2020.
  64. Macroscopic and microscopic deformation coupling in up-sampled cloth simulation. Computer Animation and Virtual Worlds, 25(3-4):435–444, 2014.
  65. Learning to simulate complex physics with graph networks. In Proceedings of the 37th International Conference on Machine Learning, pages 8459–8468. PMLR, 2020.
  66. Snug: Self-supervised neural dynamic garments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8140–8150, 2022.
  67. Spnets: Differentiable fluid dynamics for deep neural networks. In Conference on Robot Learning, pages 317–335. PMLR, 2018.
  68. High-order differentiable autoencoder for nonlinear model reduction. ACM Trans. Graph., 40(4), 2021.
  69. Loss-attentional physics-informed neural networks. Journal of Computational Physics, page 112781, 2024.
  70. Diffsdfsim: Differentiable rigid-body dynamics with implicit shapes. In 2021 International Conference on 3D Vision (3DV), pages 96–105. IEEE, 2021.
  71. Diffxpbd: Differentiable position-based simulation of compliant constraint dynamics. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 6(3):1–14, 2023.
  72. Discrete bending forces and their jacobians. Graphical models, 75(6):362–370, 2013.
  73. A simple approach to nonlinear tensile stiffness for accurate cloth simulation. ACM Transactions on Graphics, 28(4):Article–No, 2009.
  74. Huamin Wang. Gpu-based simulation of cloth wrinkles at submillimeter levels. ACM Transactions on Graphics (TOG), 40(4):1–14, 2021.
  75. Data-driven elastic models for cloth: modeling and measurement. ACM transactions on graphics (TOG), 30(4):1–12, 2011.
  76. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55:361–387, 2016.
  77. Fluidlab: A differentiable environment for benchmarking complex fluid manipulation. arXiv preprint arXiv:2303.02346, 2023.
  78. B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
  79. Learning-based cloth material recovery from video. In Proceedings of the IEEE International Conference on Computer Vision, pages 4383–4393, 2017.
  80. Human trajectory prediction via neural social physics. In The European Conference on Computer Vision (ECCV), pages 376–394, 2022.
  81. Human trajectory forecasting with explainable behavioral uncertainty. arXiv preprint arXiv:2307.01817, 2023.
  82. Meshingnet: A new mesh generation method based on deep learning. In Computational Science – ICCS 2020, pages 186–198, Cham, 2020. Springer International Publishing.
  83. MeshingNet3D: Efficient generation of adapted tetrahedral meshes for computational mechanics. Advances in Engineering Software, 157-158, 2021.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up