Papers
Topics
Authors
Recent
Search
2000 character limit reached

Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluation

Published 3 Mar 2024 in cs.CV and cs.GR | (2403.01619v1)

Abstract: Existing 3D mesh shape evaluation metrics mainly focus on the overall shape but are usually less sensitive to local details. This makes them inconsistent with human evaluation, as human perception cares about both overall and detailed shape. In this paper, we propose an analytic metric named Spectrum Area Under the Curve Difference (SAUCD) that demonstrates better consistency with human evaluation. To compare the difference between two shapes, we first transform the 3D mesh to the spectrum domain using the discrete Laplace-Beltrami operator and Fourier transform. Then, we calculate the Area Under the Curve (AUC) difference between the two spectrums, so that each frequency band that captures either the overall or detailed shape is equitably considered. Taking human sensitivity across frequency bands into account, we further extend our metric by learning suitable weights for each frequency band which better aligns with human perception. To measure the performance of SAUCD, we build a 3D mesh evaluation dataset called Shape Grading, along with manual annotations from more than 800 subjects. By measuring the correlation between our metric and human evaluation, we demonstrate that SAUCD is well aligned with human evaluation, and outperforms previous 3D mesh metrics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. https://www.gobotree.com/.
  2. Sketchfab - the best 3d viewer on the web. https://sketchfab.com/.
  3. Learning representations and generative models for 3d point clouds. In ICML, pages 40–49, 2018.
  4. Fostering generalization in single-view 3d reconstruction by learning a hierarchy of local and global shape priors. In CVPR, pages 15880–15889, 2021.
  5. Fast differentiable sorting and ranking. In ICML, pages 950–959, 2020.
  6. Discrete differential geometry. Springer, 2008.
  7. Gunilla Borgefors. Distance transformations in arbitrary dimensions. Computer vision, graphics, and image processing, pages 321–345, 1984.
  8. Robert Bridson. Fast poisson disk sampling in arbitrary dimensions. SIGGRAPH sketches, page 1, 2007.
  9. RECOMMENDATION ITU-R BT. Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, 2002.
  10. Assessing visual quality of 3-d polygonal models. IEEE Signal Processing Magazine, pages 80–90, 2011.
  11. Applied differential geometry. Cambridge University Press, 1985.
  12. Learning a general clause-to-clause relationships for enhancing emotion-cause pair extraction. arXiv preprint arXiv:2208.13549, 2022.
  13. Decor-gan: 3d shape detailization by conditional refinement. In CVPR, pages 15740–15749, 2021.
  14. Fan RK Chung. Spectral graph theory. American Mathematical Soc., 1997.
  15. MeshLab: an Open-Source Mesh Processing Tool. In Eurographics Italian Chapter Conference, 2008.
  16. Perceptual metrics for static and dynamic triangle meshes. In Comput. Graph. Forum, pages 101–125, 2013.
  17. A Modern Introduction to Probability and Statistics: Understanding why and how. Springer, 2005.
  18. Fourier analysis. American Mathematical Soc., 2001.
  19. Zhongpai Gao. Learning continuous mesh representation with spherical implicit surface. In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), pages 1–8. IEEE, 2023.
  20. Semi-supervised 3d face representation learning from unconstrained photo collections. In CVPRW, pages 348–349, 2020.
  21. Surface simplification using quadric error metrics. In SIGGRAPH, pages 209–216, 1997.
  22. Local deep implicit functions for 3d shape. In CVPR, pages 4857–4866, 2020.
  23. Self-supervised human mesh recovery with cross-representation alignment. In ECCV, pages 212–230, 2022.
  24. Progressive multi-view human mesh recovery with self-supervision. In AAAI, pages 676–684, 2023.
  25. Learning to generate and reconstruct 3d meshes with only 2d supervision. arXiv preprint arXiv:1807.09259, 2018.
  26. Matrix analysis. Cambridge University Press, 2012.
  27. Self-supervised 3d mesh reconstruction from single images. In CVPR, pages 6002–6011, 2021.
  28. Large scale multi-view stereopsis evaluation. In CVPR, pages 406–413, 2014.
  29. Poisson surface reconstruction. In Eurographics Symposium on Geometry Processing, 2006.
  30. Maurice George Kendall et al. The advanced theory of statistics. The advanced theory of statistics, 1946.
  31. Convolutional mesh regression for single-image human shape reconstruction. In CVPR, pages 4501–4510, 2019.
  32. Fast image deconvolution using hyper-laplacian priors. NeurIPS, 22, 2009.
  33. Auto-refining 3d mesh reconstruction algorithm from limited angle depth data. IEEE Access, pages 87083–87098, 2022.
  34. On information and sufficiency. The annals of mathematical statistics, pages 79–86, 1951.
  35. Guillaume Lavoué. A local roughness measure for 3d meshes and its application to visual masking. ACM Transactions on Applied Perception, pages 1–23, 2009.
  36. 3d mesh reconstruction of indoor scenes from a single image in-the-wild. In International Conference on Graphics and Image Processing, pages 457–465, 2022.
  37. Min-max similarity: A contrastive semi-supervised deep learning network for surgical tools segmentation. IEEE TMI, 2023.
  38. Pc-hmr: Pose calibration for 3d human mesh recovery from 2d images/videos. In AAAI, pages 2269–2276, 2021.
  39. High fidelity 3d hand shape reconstruction via scalable graph frequency decomposition. In CVPR, pages 16795–16804, 2023.
  40. Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and mathematics III, pages 35–57. Springer, 2003.
  41. Deephandmesh: A weakly-supervised deep encoder-decoder framework for high-fidelity hand mesh modeling. In ECCV, 2020.
  42. Total3dunderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. In CVPR, pages 55–64, 2020.
  43. Data-driven volumetric image generation from surface structures using a patient-specific deep leaning model. arXiv preprint arXiv:2304.14594, 2023a.
  44. Cycle-guided denoising diffusion probability model for 3d cross-modality mri synthesis. arXiv preprint arXiv:2305.00042, 2023b.
  45. Local laplacian filters: edge-aware image processing with a laplacian pyramid. ACM TOG, page 68, 2011.
  46. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 32, 2019.
  47. Karl Pearson. Notes on the history of correlation. Biometrika, pages 25–45, 1920.
  48. Poisson image editing. In SIGGRAPH, pages 313–318, 2003.
  49. Tid2008-a database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics, pages 30–45, 2009.
  50. Learning delaunay surface elements for mesh reconstruction. In CVPR, pages 22–31, 2021.
  51. Embodied hands: Modeling and capturing hands and bodies together. SIGGRAPH, 2017.
  52. Automated line labelling: Dataset for contour detection and 3d reconstruction. In WACV, pages 3136–3145, 2023.
  53. Gradient based image completion by solving the poisson equation. Computers & Graphics, pages 119–126, 2007.
  54. Meshmvs: Multi-view stereo guided mesh reconstruction. In 3DV, pages 1290–1300. IEEE, 2021.
  55. 3d point cloud generative adversarial network based on tree structured graph convolutions. In ICCV, pages 3859–3868, 2019.
  56. Charles Spearman. Correlation calculated from faulty data. British journal of psychology, page 271, 1910.
  57. A fast method for image noise estimation using laplacian operator and adaptive edge detection. In International Symposium on Communications, Control and Signal Processing, pages 1077–1081, 2008.
  58. Efficientnet: Improving accuracy and efficiency through automl and model scaling. arXiv preprint arXiv:1905.11946, 2019.
  59. Point scene understanding via disentangled instance mesh reconstruction. In ECCV, pages 684–701, 2022.
  60. What do single-view 3d reconstruction networks learn? In CVPR, pages 3405–3414, 2019.
  61. Gabriel Taubin. Curve and surface smoothing without shrinkage. In ICCV, pages 852–857, 1995.
  62. A benchmark for 3d mesh watermarking. In Shape Modeling International Conference, pages 231–235. IEEE, 2010.
  63. Faceverse: a fine-grained and detail-controllable 3d face morphable model from a hybrid dataset. In CVPR, pages 20333–20342, 2022.
  64. Pixel2mesh: Generating 3d mesh models from single rgb images. In ECCV, pages 52–67, 2018.
  65. Xin Wang. Laplacian operator-based edge detectors. IEEE TPAMI, pages 886–890, 2007.
  66. Deep hybrid self-prior for full 3d mesh generation. In ICCV, pages 5805–5814, 2021.
  67. Stephen Wolfram. Mathematica: a system for doing mathematics by computer. Addison Wesley Longman Publishing Co., Inc., 1991.
  68. Learning to generate 3d shapes from a single example. arXiv preprint arXiv:2208.02946, 2022.
  69. Multimodal shape completion via conditional generative adversarial networks. In ECCV, pages 281–296, 2020.
  70. Shisrcnet: Super-resolution and classification network for low-resolution breast cancer histopathology image. MICCAI, 2024.
  71. Reducing positional variance in cross-sectional abdominal ct slices with deep conditional generative models. In MICCAI, pages 202–212, 2022.
  72. Deep conditional generative models for longitudinal single-slice abdominal computed tomography harmonization. arXiv preprint arXiv:2309.09392, 2023.
  73. Cd2: Fine-grained 3d mesh reconstruction with twice chamfer distance. arXiv preprint arXiv:2206.00447, 2022.
  74. Two-stream consensus network for weakly-supervised temporal action localization. In ECCV, pages 37–54, 2020.
  75. Language-guided human motion synthesis with atomic actions. In ACM MM, pages 5262–5271, 2023a.
  76. Towards generic image manipulation detection with weakly-supervised self-consistency learning. In ICCV, pages 22390–22400, 2023b.
  77. Task-oriented low-dose ct image denoising. In MICCAI, pages 441–450, 2021.
  78. Parametric chamfer alignment based on mesh deformation. Measurement and Control, pages 192–201, 2023.
  79. Enriching local and global contexts for temporal action localization. In ICCV, pages 13516–13525, 2021.
  80. Learning disentangled classification and localization representations for temporal action localization. In AAAI, pages 3644–3652, 2022.
  81. Unsupervised 3d human mesh recovery from noisy point clouds. arXiv preprint arXiv:2107.07539, 2021.
Citations (7)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free