3D-FUTURE: 3D Furniture shape with TextURE (2009.09633v1)

Abstract: The 3D CAD shapes in current 3D benchmarks are mostly collected from online model repositories. Thus, they typically have insufficient geometric details and less informative textures, making them less attractive for comprehensive and subtle research in areas such as high-quality 3D mesh and texture recovery. This paper presents 3D Furniture shape with TextURE (3D-FUTURE): a richly-annotated and large-scale repository of 3D furniture shapes in the household scenario. At the time of this technical report, 3D-FUTURE contains 20,240 clean and realistic synthetic images of 5,000 different rooms. There are 9,992 unique detailed 3D instances of furniture with high-resolution textures. Experienced designers developed the room scenes, and the 3D CAD shapes in the scene are used for industrial production. Given the well-organized 3D-FUTURE, we provide baseline experiments on several widely studied tasks, such as joint 2D instance segmentation and 3D object pose estimation, image-based 3D shape retrieval, 3D object reconstruction from a single image, and texture recovery for 3D shapes, to facilitate related future researches on our database.

Overview of the 3D-FUTURE Dataset

The paper presents 3D-FUTURE, an extensive 3D furniture dataset specifically designed for research in comprehensive and subtle recovery of high-quality 3D shapes and textures. This dataset addresses the limitations of existing 3D benchmarks that primarily consist of CAD shapes from online repositories, which often lack geometric refinement and detailed textures, thus hindering their applicability in advanced 3D computer vision and graphics research.

Core Contributions and Features

1. Dataset Composition:
   • Scale and Diversity: 3D-FUTURE offers 20,240 synthetic images across 5,000 rooms, with 9,992 unique 3D furniture models. These models feature high-resolution textures and detailed geometric properties suitable for industrial use.
   • Annotation and Alignments: The dataset is richly annotated, including instance-level semantic annotations and alignments between 2D images and 3D models, facilitating research across different fields such as joint 2D instance segmentation and 3D object pose estimation, as well as texture and mesh recovery.
2. Industrial Relevance:
   • The 3D models are derived from industry-grade CAD designs, ensuring modern relevance and applicability.
   • The dataset bridges the gap between academic research and industrial production by providing high-quality shapes used in real-world applications.
3. Design Innovations:
   • Furnishing Suit Composition: A system for generating aesthetically pleasing room designs using AI-driven compatibility checks complemented by designer reviews to ensure visual coherence and quality.
   • Efficient Design Process: By utilizing a blend of machine learning techniques and a vast pool of professional design data, the process of creating detailed and appealing room setups is streamlined.

Baseline Experiments

The paper reports several baseline experiments to demonstrate the dataset's potential across different tasks:

• 3D Object Recognition: Using well-known architectures like MVCNN and PointNet++, the dataset challenges existing recognition methods to work with fine-grained categories, highlighting areas for improvement in existing 3D recognition models.
• Image-based 3D Shape Retrieval: The dataset's comprehensive 2D-3D alignments enable thorough cross-domain retrieval studies, showcasing its utility in bridging image data with 3D models.
• Joint Instance Segmentation and Pose Estimation: Explores the synergistic prediction of object masks and 6DoF poses, pushing the boundaries of joint 3D and 2D tasks which are crucial for robotics and AR applications.
• 3D Object Reconstruction: Evaluated using state-of-the-art methods, demonstrating the challenges presented by detailed shapes and textures within the dataset.
• Texture Synthesis: Analyzed leveraging frameworks like Texture Fields and a novel BicycleGAN++, aiming at realistic texture recovery—a crucial aspect of reconciling virtual objects with their real-world counterparts.

Implications and Future Directions

The 3D-FUTURE dataset sets a new standard for the evaluation and development of algorithms in the 3D vision domain. It effectively bridges existing gaps by providing fine-grained and comprehensive data that adheres to industrial standards while enabling a plethora of fundamental and novel research opportunities. The intricacy and scale of the dataset suggest numerous avenues for future research, particularly in the field of AI-driven design processes, high-quality reconstruction, and cross-domain retrieval systems. As 3D-FUTURE serves to cultivate advances in both theoretical research and practical applications, it is poised to significantly impact the development of more robust, accurate, and scalable 3D vision technologies.