BézierSketch: A generative model for scalable vector sketches (2007.02190v2)

Abstract: The study of neural generative models of human sketches is a fascinating contemporary modeling problem due to the links between sketch image generation and the human drawing process. The landmark SketchRNN provided breakthrough by sequentially generating sketches as a sequence of waypoints. However this leads to low-resolution image generation, and failure to model long sketches. In this paper we present B\'ezierSketch, a novel generative model for fully vector sketches that are automatically scalable and high-resolution. To this end, we first introduce a novel inverse graphics approach to stroke embedding that trains an encoder to embed each stroke to its best fit B\'ezier curve. This enables us to treat sketches as short sequences of paramaterized strokes and thus train a recurrent sketch generator with greater capacity for longer sketches, while producing scalable high-resolution results. We report qualitative and quantitative results on the Quick, Draw! benchmark.

• The paper introduces an inverse graphics technique that encodes sketch strokes as smooth Bezier curves for scalable vector representation.
• The model leverages a neural encoder and differentiable Bezier decoder to reconstruct strokes and reduce noise compared to SketchRNN.
• Experimental results on the Quick, Draw! dataset show improved resolution and lower FID scores, highlighting its efficiency in sketch generation.

Overview of "BezierSketch: A Generative Model for Scalable Vector Sketches"

The paper "BezierSketch: A Generative Model for Scalable Vector Sketches" introduces a new approach to the generative modeling of sketches, addressing some of the limitations inherent in previous methods such as SketchRNN. Specifically, the authors propose a novel inverse graphics technique for encoding sketches as sequences of parameterized Bezier curves, providing several advantages in terms of scalability, resolution, and model capacity.

Technical Contributions

1. Bezier Curve Representation: The authors introduce the use of Bezier curves to represent sketch strokes. This involves encoding each stroke of a sketch as a smooth, parametric curve, allowing the generative model to operate with a concise and efficient vector-based representation rather than sequences of pixels.
2. Inverse Graphics Model: A key component of the methodology is a vision-as-inverse-graphics approach, integrating a neural encoder to map human sketch strokes to Bezier curve parameters. A differentiable Bezier decoder is employed to reconstruct strokes, enabling the training of the encoder via reconstruction loss.
3. Generative Model: The paper details a new sequential generative model using the Bezier representation. This model outperforms SketchRNN by generating longer sketches efficiently and creating scalable, high-resolution vector graphics that are ideal for digital art applications.

Results and Discussion

The model was tested on the Quick, Draw! Benchmark, a large-scale dataset of free-hand sketches. The experiments reported both qualitative improvements in sketch appearance and quantitative gains in performance metrics. Specifically, the proposed method showed:

• Enhanced Scalability: By implementing the Bezier curve approach, the representation of sketches became inherently scalable, allowing them to be rendered at arbitrary resolutions without degradation.
• Reduced Noise and Improved Capacity: The fixed-length stroke embeddings significantly reduce sampling noise and allow the underlying recurrent neural network to utilize its capacity more effectively, especially for longer sketches.
• Improved Metric Performance: The paper presents a modified Fréchet Inception Distance (FID) score to quantify the quality of generated sketches, showing lower scores (indicating better performance) than the SketchRNN, particularly for longer sketches.

Implications and Future Directions

The implications of this work extend to various applications where efficient, high-quality vector graphics are essential, such as digital illustration, animation, and computer-aided design (CAD). The generative model's ability to create infinite-resolution graphics could transform how digital content is created, potentially impacting fields beyond traditional sketching.

Future avenues for research proposed in the paper include exploring more complex parametric curves, like B-splines, and integrating raster-based image inputs directly into the generative workflow. These enhancements could further increase the model's flexibility and application range, opening new frontiers in AI-driven graphic design and generative art.

Through its innovative use of Bezier curves and inverse graphics, the paper advances the state of the art in sketch generation, proposing a robust framework with significant practical and theoretical benefits for scalable vector graphic creation.