Single-image 3D reconstruction from learned generative 3D priors

Establish methods for reconstructing 3D objects from a single image or a small number of images by leveraging a learned generative prior over 3D objects, addressing the feasibility and requirements of this approach under limited-view constraints.

Background

The paper contrasts reconstruction methods that regress geometry from images with generative methods that synthesize canonical 3D objects. Reconstruction approaches often recover only visible regions and operate in a view-centric frame, struggling with occluded or unseen areas and typically not producing canonical models or object-centric camera poses. Generative methods can produce canonically posed objects and strong priors but frequently lack faithful adherence to a given input image and do not explicitly estimate pose, making post-hoc alignment costly and fragile.

Within this context, achieving single-image (or few-image) object reconstruction using a learned generative 3D prior is identified as an unresolved challenge. The work introduces Cupid to address this gap, but the general problem of reliably performing reconstruction from minimal views by conditioning on generative 3D distributions remains open, motivating further research into algorithms and training regimes that reconcile generative priors with observation constraints.