DrawingSpinUp: 3D Animation from Single Character Drawings (2409.08615v1)

Abstract: Animating various character drawings is an engaging visual content creation task. Given a single character drawing, existing animation methods are limited to flat 2D motions and thus lack 3D effects. An alternative solution is to reconstruct a 3D model from a character drawing as a proxy and then retarget 3D motion data onto it. However, the existing image-to-3D methods could not work well for amateur character drawings in terms of appearance and geometry. We observe the contour lines, commonly existing in character drawings, would introduce significant ambiguity in texture synthesis due to their view-dependence. Additionally, thin regions represented by single-line contours are difficult to reconstruct (e.g., slim limbs of a stick figure) due to their delicate structures. To address these issues, we propose a novel system, DrawingSpinUp, to produce plausible 3D animations and breathe life into character drawings, allowing them to freely spin up, leap, and even perform a hip-hop dance. For appearance improvement, we adopt a removal-then-restoration strategy to first remove the view-dependent contour lines and then render them back after retargeting the reconstructed character. For geometry refinement, we develop a skeleton-based thinning deformation algorithm to refine the slim structures represented by the single-line contours. The experimental evaluations and a perceptual user study show that our proposed method outperforms the existing 2D and 3D animation methods and generates high-quality 3D animations from a single character drawing. Please refer to our project page (https://lordliang.github.io/DrawingSpinUp) for the code and generated animations.

• The paper introduces a dual-stage process that first removes and later restores contour lines to convert 2D drawings into accurate 3D animated models.
• It employs a skeleton-based thinning deformation algorithm to refine slender structures, ensuring the faithful portrayal of delicate features.
• Experimental evaluations show superior motion consistency (4.55) and style preservation (4.53), validating its effectiveness over previous methods.

DrawingSpinUp: 3D Animation from Single Character Drawings

The paper "DrawingSpinUp: 3D Animation from Single Character Drawings" by Zhou et al. focuses on the task of converting 2D character drawings into 3D animations, addressing limitations in existing methods that reduce them to flat 2D motions. The proposed system, DrawingSpinUp, introduces innovative techniques to bridge the gap between amateur character drawings and high-quality 3D models suitable for dynamic animations.

The authors identify key challenges in transforming 2D character drawings, particularly amateur ones, into 3D animations: reconciling view-dependent contour lines and mitigating difficulty in reconstructing delicate structures such as slim limbs. Existing image-to-3D methods often fail on amateur drawings due to the significant ambiguity introduced by view-dependent contours and the intricacy required to handle single-line depictions of thin structures.

To overcome these challenges, DrawingSpinUp employs a dual-stage process that involves:

1. Contour Removal and Restoration: The system initially removes view-dependent contour lines using a novel image-to-image translation network based on FFC-ResNet. After generating the 3D model and applying target motions, the contours are restored using a geometry-aware stylization network that ensures consistency with the input drawing’s style. This "removal-then-restoration" strategy preserves the visual fidelity of the original drawing while enabling accurate 3D reconstruction.
2. Skeleton-based Thinning Deformation: To refine slim structures depicted by single-line contours, the authors introduce a thinning deformation algorithm. This method employs a bi-harmonic interpolation process guided by skeleton-based handles to ensure that slender parts like limbs are accurately represented in the 3D model without excessive thickening, which is a common failure in preceding techniques.

Experimental evaluation and user studies provided empirical support for the method’s efficacy. Key numerical results include significantly higher ratings for both motion consistency and style preservation, with the system averaging 4.55 and 4.53 out of 5, respectively. These results indicate the system's superiority in generating animations that closely adhere to input motions while maintaining the character’s original artistic style.

The implications of DrawingSpinUp are profound for both practical applications and theoretical advancements. Practically, this system can democratize 3D animation creation, enabling amateurs and professionals alike to generate complex animations from simple drawings. Theoretically, DrawingSpinUp advances the integration of non-photorealistic rendering techniques with 3D geometry processing, offering a modular approach that could inspire future research in hybrid 2D-3D transformation pipelines.

Anticipated future developments in AI, influenced by this research, may involve improving real-time performance for such systems, refining contour restoration for highly abstract or self-occluded drawings, and extending the domain to accommodate a wider variety of artistic styles and subject forms, including quadrupeds. Furthermore, integrating text prompts could open new avenues for interactive storytelling and immersive content creation in gaming and virtual/augmented reality (VR/AR) environments.

DrawingSpinUp represents a significant step toward more versatile and accessible tools for creative professionals and hobbyists, paving the way for more intuitive and high-fidelity conversions of 2D artistic expressions into the dynamic world of 3D animation.