Interactive 3D Object Generation and Previewing

Establish an interactive 3D object generation and previewing workflow that provides fine-grained controllability during the generative process and delivers responsive feedback without slow reconstruction, thereby overcoming the current limitations of limited control and reconstruction latency in 3D generative modeling.

Background

The paper addresses the challenge of bringing 3D-aware controllability and interactivity to 3D asset generation. While 2D diffusion models support control and iterative local edits (e.g., inpainting), analogous capabilities in 3D have been lacking due to limitations in existing methods’ controllability and the need for time-consuming reconstruction before previewing edits.

The authors explicitly note that interactive generation and previewing for 3D objects is not yet solved, highlighting the need for techniques that can support fine-grained control and rapid, responsive preview to enable practical and productive 3D modeling workflows.