Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dr.Hair: Reconstructing Scalp-Connected Hair Strands without Pre-training via Differentiable Rendering of Line Segments (2403.17496v2)

Published 26 Mar 2024 in cs.CV and cs.GR

Abstract: In the film and gaming industries, achieving a realistic hair appearance typically involves the use of strands originating from the scalp. However, reconstructing these strands from observed surface images of hair presents significant challenges. The difficulty in acquiring Ground Truth (GT) data has led state-of-the-art learning-based methods to rely on pre-training with manually prepared synthetic CG data. This process is not only labor-intensive and costly but also introduces complications due to the domain gap when compared to real-world data. In this study, we propose an optimization-based approach that eliminates the need for pre-training. Our method represents hair strands as line segments growing from the scalp and optimizes them using a novel differentiable rendering algorithm. To robustly optimize a substantial number of slender explicit geometries, we introduce 3D orientation estimation utilizing global optimization, strand initialization based on Laplace's equation, and reparameterization that leverages geometric connectivity and spatial proximity. Unlike existing optimization-based methods, our method is capable of reconstructing internal hair flow in an absolute direction. Our method exhibits robust and accurate inverse rendering, surpassing the quality of existing methods and significantly improving processing speed.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (72)
  1. Insightface, 2023. https://github.com/deepinsight/insightface.
  2. Agisoft. Metashape, 2023. https://www.agisoft.com/.
  3. Autodesk. Maya xgen, 2023. https://www.autodesk.com/products/maya/.
  4. Blender. Hair nodes, 2023. https://docs.blender.org/manual/en/3.6/modeling/geometry_nodes/hair/index.html.
  5. Authentic volumetric avatars from a phone scan. ACM Trans. Graph., 41(4), 2022.
  6. A class of local interpolating splines. In Computer aided geometric design, pages 317–326. Elsevier, 1974.
  7. Dan Cernea. OpenMVS: Multi-view stereo reconstruction library. https://cdcseacave.github.io/openMVS, 2020.
  8. Dynamic hair manipulation in images and videos. ACM Trans. Graph., 32(4), 2013.
  9. Autohair: Fully automatic hair modeling from a single image. ACM Trans. Graph., 35(4), 2016.
  10. Knn matting. IEEE transactions on pattern analysis and machine intelligence, 35(9):2175–2188, 2013.
  11. Differentiable surface rendering via non-differentiable sampling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6088–6097, 2021.
  12. Ephere. Ornatrix for maya, 2023. https://ephere.com/plugins/autodesk/maya/ornatrix/.
  13. Robust hair capture using simulated examples. ACM Trans. Graph., 33(4), 2014a.
  14. Capturing braided hairstyles. ACM Trans. Graph., 33(6), 2014b.
  15. Single-view hair modeling using a hairstyle database. ACM Trans. Graph., 34(4), 2015.
  16. Simulation-Ready Hair Capture. Computer Graphics Forum, 2017a.
  17. Avatar digitization from a single image for real-time rendering. ACM Trans. Graph., 36(6), 2017b.
  18. Capturing hair assemblies fiber by fiber. ACM Trans. Graph., 28(5):1–9, 2009.
  19. Mitsuba 3 renderer, 2022. https://mitsuba-renderer.org.
  20. Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, page 0, 2006.
  21. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023.
  22. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  23. Deepmvshair: Deep hair modeling from sparse views. In SIGGRAPH Asia 2022 Conference Papers, New York, NY, USA, 2022. Association for Computing Machinery.
  24. Modular primitives for high-performance differentiable rendering. ACM Transactions on Graphics, 39(6), 2020.
  25. Cdgnet: Class distribution guided network for human parsing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4473–4482, 2022.
  26. Soft rasterizer: A differentiable renderer for image-based 3d reasoning. The IEEE International Conference on Computer Vision (ICCV), 2019.
  27. Neural volumes: Learning dynamic renderable volumes from images. ACM Trans. Graph., 38(4):65:1–65:14, 2019.
  28. Mixture of volumetric primitives for efficient neural rendering. ACM Transactions on Graphics (ToG), 40(4):1–13, 2021.
  29. Gaussianhair: Hair modeling and rendering with light-aware gaussians. arXiv preprint arXiv:2402.10483, 2024.
  30. Dynamic hair capture. Rapp. tech. Princeton University, 2011.
  31. Multi-view hair capture using orientation fields. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 1490–1497. IEEE, 2012.
  32. Structure-aware hair capture. ACM Trans. Graph., 32(4), 2013.
  33. Refinement of hair geometry by strand integration. Computer Graphics Forum (proceedings of Pacific Graphics), 42(7), 2023.
  34. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, pages 405–421. Springer, 2020.
  35. Strand-accurate multi-view hair capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  36. Large steps in inverse rendering of geometry. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), 40(6), 2021.
  37. Mitsuba 2: A retargetable forward and inverse renderer. Transactions on Graphics (Proceedings of SIGGRAPH Asia), 38(6), 2019.
  38. Capture of hair geometry from multiple images. ACM transactions on graphics (TOG), 23(3):712–719, 2004.
  39. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph., 27(3):1–9, 2008.
  40. Poisson image editing. ACM Trans. Graph., 22(3):313–318, 2003.
  41. Robert Clay Prim. Shortest connection networks and some generalizations. The Bell System Technical Journal, 36(6):1389–1401, 1957.
  42. H3d-net: Few-shot high-fidelity 3d head reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5620–5629, 2021.
  43. Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501, 2020.
  44. Neural strands: Learning hair geometry and appearance from multi-view images. In ECCV, 2022.
  45. Adop: Approximate differentiable one-pixel point rendering. ACM Trans. Graph., 41(4), 2022.
  46. 3d hair synthesis using volumetric variational autoencoders. ACM Trans. Graph., 37(6), 2018.
  47. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  48. Deepsketchhair: Deep sketch-based 3d hair modeling. IEEE transactions on visualization and computer graphics, 27(7):3250–3263, 2020.
  49. Ct2hair: High-fidelity 3d hair modeling using computed tomography. ACM Transactions on Graphics, 42(4):1–13, 2023.
  50. Neural haircut: Prior-guided strand-based hair reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 19762–19773, 2023a.
  51. Haar: Text-conditioned generative model of 3d strand-based human hairstyles. ArXiv, 2023b.
  52. Il’ya Meerovich Sobol’. On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal Vychislitel’noi Matematiki i Matematicheskoi Fiziki, 7(4):784–802, 1967.
  53. Human Hair Inverse Rendering using Multi-View Photometric data. In Eurographics Symposium on Rendering - DL-only Track. The Eurographics Association, 2021.
  54. Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization. Computer Graphics Forum, 2022.
  55. Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence, 13(04):376–380, 1991.
  56. Example-based hair geometry synthesis. In ACM SIGGRAPH 2009 Papers, New York, NY, USA, 2009. Association for Computing Machinery.
  57. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. NeurIPS, 2021.
  58. Hvh: Learning a hybrid neural volumetric representation for dynamic hair performance capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6143–6154, 2022.
  59. Neuwigs: A neural dynamic model for volumetric hair capture and animation. In CVPR, pages 8641–8651, 2023.
  60. Modeling hair from multiple views. In ACM SIGGRAPH 2005 Papers, page 816–820, New York, NY, USA, 2005. Association for Computing Machinery.
  61. Neuralhdhair: Automatic high-fidelity hair modeling from a single image using implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1526–1535, 2022.
  62. Dynamic hair modeling from monocular videos using deep neural networks. ACM Trans. Graph., 38(6), 2019.
  63. Differentiable surface splatting for point-based geometry processing. ACM Trans. Graph., 38(6), 2019.
  64. Advanced techniques in real-time hair rendering and simulation. In ACM SIGGRAPH 2010 Courses, New York, NY, USA, 2010. Association for Computing Machinery.
  65. Hair meshes. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2009), 28(5):166:1–166:7, 2009.
  66. Hair-gan: Recovering 3d hair structure from a single image using generative adversarial networks. Visual Informatics, 3(2):102–112, 2019.
  67. A data-driven approach to four-view image-based hair modeling. ACM Trans. Graph., 36(4), 2017.
  68. Modeling hair from an rgb-d camera. ACM Trans. Graph., 37(6), 2018.
  69. Energyhair: Sketch-based interactive guide hair design using physics-inspired energy. In Graphics Interface 2022, 2022.
  70. Hairstep: Transfer synthetic to real using strand and depth maps for single-view 3d hair modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12726–12735, 2023.
  71. Hairnet: Single-view hair reconstruction using convolutional neural networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 235–251, 2018.
  72. Groomgen: A high-quality generative hair model using hierarchical latent representations. ACM Trans. Graph., 42(6), 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets