3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow (2203.15190v1)

Abstract: Reconstructing 3D shape from a single 2D image is a challenging task, which needs to estimate the detailed 3D structures based on the semantic attributes from 2D image. So far, most of the previous methods still struggle to extract semantic attributes for 3D reconstruction task. Since the semantic attributes of a single image are usually implicit and entangled with each other, it is still challenging to reconstruct 3D shape with detailed semantic structures represented by the input image. To address this problem, we propose 3DAttriFlow to disentangle and extract semantic attributes through different semantic levels in the input images. These disentangled semantic attributes will be integrated into the 3D shape reconstruction process, which can provide definite guidance to the reconstruction of specific attribute on 3D shape. As a result, the 3D decoder can explicitly capture high-level semantic features at the bottom of the network, and utilize low-level features at the top of the network, which allows to reconstruct more accurate 3D shapes. Note that the explicit disentangling is learned without extra labels, where the only supervision used in our training is the input image and its corresponding 3D shape. Our comprehensive experiments on ShapeNet dataset demonstrate that 3DAttriFlow outperforms the state-of-the-art shape reconstruction methods, and we also validate its generalization ability on shape completion task.

• The paper proposes 3DAttriFlow to disentangle semantic attributes, enhancing the reconstruction of detailed 3D shapes from 2D images.
• It employs a hierarchical decoder that integrates high-level semantic features with low-level details to improve reconstruction accuracy.
• Experiments on the ShapeNet dataset confirm that 3DAttriFlow outperforms state-of-the-art methods and boosts shape completion tasks without extra labeling.

The paper, "3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow," addresses the significant challenge of reconstructing detailed 3D shapes from single 2D images. This task requires the accurate estimation of 3D structures based on the semantic attributes present in the 2D image. Previous methods have struggled with extracting these semantic attributes, primarily because they are often implicit and entangled within the image data.

To overcome these challenges, the authors propose a novel approach called 3DAttriFlow. This method aims to disentangle and extract semantic attributes at different levels of semantics within the input image. The key innovation here is the integration of these disentangled semantic attributes into the 3D shape reconstruction process. Disentangling offers clear guidance for reconstructing precise attributes of the 3D shape.

In practical terms, the 3D decoder within 3DAttriFlow explicitly captures high-level semantic features at the lower layers of the network and utilizes low-level features at the network's top layers. This hierarchical processing allows the model to reconstruct more accurate and detailed 3D shapes. A noteworthy aspect of 3DAttriFlow is that it learns the explicit disentangling without requiring additional labels; the only supervision during training is the input image coupled with its corresponding 3D shape.

The authors validate their approach through comprehensive experiments conducted on the ShapeNet dataset. The results reveal that 3DAttriFlow significantly outperforms existing state-of-the-art shape reconstruction methods. Moreover, the authors also test the generalization ability of 3DAttriFlow on a shape completion task, further demonstrating its robustness and effectiveness.

In summary, the paper contributes a new method for 3D shape reconstruction that effectively addresses the disentanglement and integration of semantic attributes from 2D images, leading to more accurate and detailed 3D reconstructions without the need for extra labeling.