Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction (2306.05872v2)

Published 9 Jun 2023 in cs.CV and cs.GR

Abstract: Generating realistic human 3D reconstructions using image or video data is essential for various communication and entertainment applications. While existing methods achieved impressive results for body and facial regions, realistic hair modeling still remains challenging due to its high mechanical complexity. This work proposes an approach capable of accurate hair geometry reconstruction at a strand level from a monocular video or multi-view images captured in uncontrolled lighting conditions. Our method has two stages, with the first stage performing joint reconstruction of coarse hair and bust shapes and hair orientation using implicit volumetric representations. The second stage then estimates a strand-level hair reconstruction by reconciling in a single optimization process the coarse volumetric constraints with hair strand and hairstyle priors learned from the synthetic data. To further increase the reconstruction fidelity, we incorporate image-based losses into the fitting process using a new differentiable renderer. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. Sal: Sign agnostic learning of shapes from raw data. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2562–2571, 2019.
  2. J. E. Bresenham. Algorithm for computer control of a digital plotter. IBM Systems Journal, 4(1):25–30, 1965.
  3. How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In International Conference on Computer Vision, 2017.
  4. Authentic volumetric avatars from a phone scan. ACM Transactions on Graphics (TOG), 41:1 – 19, 2022.
  5. Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
  6. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, 2017.
  7. Autohair: fully automatic hair modeling from a single image. ACM Trans. Graph., 35:116:1–116:12, 2016.
  8. Neural ordinary differential equations, 2018.
  9. Blender Online Community. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2023.
  10. Digital geometry processing with discrete exterior calculus. In International Conference on Computer Graphics and Interactive Techniques, 2013.
  11. Improving neural implicit surfaces geometry with patch warping. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6250–6259, 2021.
  12. Acquiring the reflectance field of a human face. Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 2000.
  13. Collaborative regression of expressive bodies using moderation. In International Conference on 3D Vision (3DV), pages 792–804, Dec. 2021.
  14. Learning an animatable detailed 3D face model from in-the-wild images. volume 40, 2021.
  15. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. Advances in Neural Information Processing Systems (NeurIPS), 2022.
  16. Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8645–8654, 2020.
  17. Learning neural parametric head models. ArXiv, abs/2212.02761, 2022.
  18. Neural head avatars from monocular rgb videos. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18632–18643, 2021.
  19. Implicit geometric regularization for learning shapes. In International Conference on Machine Learning, 2020.
  20. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2015.
  21. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA, 2020. Curran Associates Inc.
  22. Headnerf: A realtime nerf-based parametric head model. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20342–20352, 2021.
  23. Single-view hair modeling using a hairstyle database. ACM Transactions on Graphics (TOG), 34:1 – 9, 2015.
  24. Accelerating 3d deep learning with pytorch3d. In SIGGRAPH Asia 2020 Courses, SA ’20, New York, NY, USA, 2020. Association for Computing Machinery.
  25. Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  26. Modnet: Real-time trimap-free portrait matting via objective decomposition. In AAAI, 2022.
  27. Realistic one-shot mesh-based head avatars. In European Conference on Computer Vision, 2022.
  28. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  29. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013.
  30. Deepmvshair: Deep hair modeling from sparse views. SIGGRAPH Asia 2022 Conference Papers, 2022.
  31. Efficient implementation of marching cubes’ cases with topological guarantees. Journal of Graphics Tools, 8:1 – 15, 2003.
  32. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):194:1–194:17, 2017.
  33. Barf: Bundle-adjusting neural radiance fields. In IEEE International Conference on Computer Vision (ICCV), 2021.
  34. On the limited memory bfgs method for large scale optimization. Mathematical Programming, 45:503–528, 1989.
  35. Cdgnet: Class distribution guided network for human parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4473–4482, June 2022.
  36. Soft rasterizer: A differentiable renderer for image-based 3d reasoning. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7707–7716, 2019.
  37. Neural volumes. ACM Transactions on Graphics (TOG), 38:1 – 14, 2019.
  38. Mixture of volumetric primitives for efficient neural rendering. ACM Transactions on Graphics (TOG), 40:1 – 13, 2021.
  39. Smpl: a skinned multi-person linear model. ACM Trans. Graph., 34:248:1–248:16, 2015.
  40. Decoupled weight decay regularization. In International Conference on Learning Representations, 2017.
  41. Human hair inverse rendering using multi-view photometric data. In Eurographics Symposium on Rendering, 2021.
  42. Modulated periodic activations for generalizable local functional representations. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14194–14203, 2021.
  43. Keypointnerf: Generalizing image-based volumetric avatars using relative spatial encoding of keypoints. In European Conference on Computer Vision, 2022.
  44. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, 2020.
  45. Strand-accurate multi-view hair capture. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 155–164, 2019.
  46. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In International Conference on Computer Vision (ICCV), 2021.
  47. STAR: A sparse trained articulated human body regressor. In European Conference on Computer Vision (ECCV), pages 598–613, 2020.
  48. SUPR: A sparse unified part-based human body model. In European Conference on Computer Vision (ECCV), 2022.
  49. Capture of hair geometry from multiple images. ACM SIGGRAPH 2004 Papers, 2004.
  50. Deepsdf: Learning continuous signed distance functions for shape representation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 165–174, 2019.
  51. Nerfies: Deformable neural radiance fields. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 5845–5854, 2020.
  52. Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields. ACM Trans. Graph., 40, 2021.
  53. Expressive body capture: 3d hands, face, and body from a single image. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10967–10977, 2019.
  54. Dreamfusion: Text-to-3d using 2d diffusion. arXiv, 2022.
  55. Npbg++: Accelerating neural point-based graphics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15969–15979, June 2022.
  56. H3d-net: Few-shot high-fidelity 3d head reconstruction. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 5600–5609, 2021.
  57. Accelerating 3d deep learning with pytorch3d. SIGGRAPH Asia 2020 Courses, 2019.
  58. Neural strands: Learning hair geometry and appearance from multi-view images. European Conference on Computer Vision (ECCV), 2022.
  59. 3d hair synthesis using volumetric variational autoencoders. ACM Transactions on Graphics (TOG), 37:1 – 12, 2018.
  60. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  61. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision (ECCV), 2016.
  62. Hand keypoint detection in single images using multiview bootstrapping. In CVPR, 2017.
  63. Implicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
  64. Blindly assess image quality in the wild guided by a self-adaptive hyper network. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3664–3673, 2020.
  65. Deep image prior. Int. J. Comput. Vis., 128(7):1867–1888, 2020.
  66. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Advances in Neural Information Processing Systems (NeurIPS), 2021.
  67. Prior-guided multi-view 3d head reconstruction. IEEE Transactions on Multimedia, 24:4028–4040, 2021.
  68. Neuwigs: A neural dynamic model for volumetric hair capture and animation. ArXiv, abs/2212.00613, 2022.
  69. Hvh: Learning a hybrid neural volumetric representation for dynamic hair performance capture. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6133–6144, 2021.
  70. Convolutional pose machines. In CVPR, 2016.
  71. OpenGL programming guide: the official guide to learning OpenGL, version 1.2. Addison-Wesley Longman Publishing Co., Inc., 1999.
  72. Bernhard P. Wrobel. Multiple view geometry in computer vision. Künstliche Intell., 15:41, 2001.
  73. Neuralhdhair: Automatic high-fidelity hair modeling from a single image using implicit neural representations. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1516–1525, 2022.
  74. Dynamic hair modeling from monocular videos using deep neural networks. ACM Transactions on Graphics (TOG), 38:1 – 12, 2019.
  75. Volume rendering of neural implicit surfaces. In Neural Information Processing Systems, 2021.
  76. Hair meshes. ACM SIGGRAPH Asia 2009 papers, 2009.
  77. Hair-gan: Recovering 3d hair structure from a single image using generative adversarial networks. Vis. Informatics, 3:102–112, 2019.
  78. I m avatar: Implicit morphable head avatars from videos. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13535–13545, 2021.
  79. Pointavatar: Deformable point-based head avatars from videos. ArXiv, abs/2212.08377, 2022.
  80. Hairnet: Single-view hair reconstruction using convolutional neural networks. In European Conference on Computer Vision, 2018.
Citations (19)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube