DesigNet: Learning to Draw Vector Graphics as Designers Do

Abstract: AI-driven content generation has made remarkable progress in recent years. However, neural networks and human designers operate in fundamentally different ways, making collaboration between them challenging. We address this gap for Scalable Vector Graphics (SVG) by equipping neural networks with tools commonly used by designers, such as axis alignment and explicit continuity control at command junctions. We introduce DesigNet, a hierarchical Transformer-VAE that operates directly on SVG sequences with a continuous command parameterization. Our main contributions are two differentiable modules: a continuity self-refinement module that predicts $C<sup>0$, $G<sup>1$, and $C<sup>1$ continuity for each curve point and enforces it by modifying Bézier control points, and an alignment self-refinement module with snapping capabilities for horizontal or vertical lines. DesigNet produces editable outlines and achieves competitive results against state-of-the-art methods, with notably higher accuracy in continuity and alignment. These properties ensure the outputs are easier to refine and integrate into professional design workflows. Source Code: https://github.com/TomasGuija/DesigNet.

• The paper presents a hierarchical Transformer-VAE that integrates differentiable self-refinement modules to enforce precise continuity and axis alignment in SVG generation.
• It employs a partitioned latent space to separate global style from local geometric details, enabling accurate one-shot font and icon synthesis.
• Experimental results show significant improvements in IoU, L1 error, and reconstruction quality over previous methods, enhancing downstream editability.

DesigNet: Hierarchical Transformer-VAE with Designer-Oriented SVG Controls

Introduction

DesigNet introduces a hierarchical Transformer-based variational autoencoder (VAE) for the generation of scalable vector graphics (SVG), targeting high-fidelity font and icon synthesis (2604.06494). The model specifically addresses deficiencies in previous SVG generation methods by closely aligning network outputs and controls with established practices and geometric intuitions used by expert designers. This is realized via two core innovations: differentiable self-refinement modules for explicit axis alignment and continuity at control point junctions, and a continuous SVG parameterization supporting smooth and precise geometric attributes. The approach delivers editable SVGs with improved continuity, alignment, and downstream editability, setting new quantitative baselines across multiple datasets, including Latin and Chinese fonts and icons.

Model Architecture and Methodology

DesigNet employs a hierarchical Transformer-VAE. At its core, the architecture processes SVG glyphs at both the contour (path) and glyph (global) levels, leveraging a partitioned latent space to preserve fine-grained geometric detail while capturing global stylistic context.

The SVG input is parsed into a continuous command-argument format, eschewing quantized coordinate tokens to prevent geometric artifacts and ensure smooth interpolation. Each drawing command is embedded by concatenating a command-type vector, a learned projection of continuous arguments, and a positional encoding. This representation is processed hierarchically: individual paths are encoded and pooled, then aggregated for glyph-level context.

The encoder produces both glyph-level and path-level latent vectors via Gaussian distributions, drawing on NVAE-inspired techniques for information partitioning. The decoder mirrors this hierarchy, supporting cross-attention between global style and local geometric details.

Figure 1: Overview of DesigNet’s process, encoding a set of reference glyphs, combining learned style with target identities, and post-processing for refined SVG output.

Differentiable Self-Refinement: Continuity and Alignment

Two differentiable modules enforce geometric regularity matching professional design practices:

1. Continuity Self-Refinement: Predicts and enforces three levels of junction continuity between curve segments—$C^0$ (geometric), $G^1$ (collinear tangents), and $C^1$ (collinear tangents and equal magnitude). The module applies deterministic, differentiable adjustments to Bézier control points to satisfy predicted continuity levels at each junction. Supervision uses a cost-sensitive cross-entropy formulation, heavily penalizing severe misclassifications.

Figure 2: Illustration of $C^0$, $G^1$, and $C^1$ continuity types at command junctions.

2. Alignment Self-Refinement: Assigns axis-alignment labels (horizontal, vertical, none) to line segments, using predicted logits to deterministically snap endpoints to axes if required. Like continuity, this operates via straight-through estimators in training, ensuring that gradients propagate through the refinement path.

Figure 3: Output glyphs before and after self-refinement—highlighting corrections to axis alignment and junction smoothness.

Both modules can be integrated into any SVG-generating model, providing explicit, supervised geometric decision variables that are critical for professional designers. Forward passes apply hard decisions, while gradients flow back via softmax relaxations.

Hierarchical Font and Icon Generation

DesigNet’s partitioned latent space supports diverse tasks, including one-shot font generation (producing an entire alphabet from a subset of reference glyphs), cross-reconstruction, and latent-space interpolation. The model is stylized by encoding reference sets (e.g., "H", "a", "m", "b", "u", "r", ...), then decoding held-out glyphs with target identity embeddings fused in the decoder.

The learned latent space supports semantically meaningful interpolations between glyphs and font styles, spanning properties such as weight and slant.

Figure 4: Examples of DesigNet’s latent space interpolation between fonts with varying weight and slant.

DesigNet is also evaluated on icon datasets, demonstrating the transferability of geometric regularity modules beyond typography.

Figure 5: Qualitative reconstructions of icon outlines: ground truth versus DesigNet outputs.

Experimental Results

Quantitative Performance

DesigNet achieves superior IoU, lower $\ell_1$ error, and reduced reconstruction error compared to DeepSVG, DeepVecFont-v2, and DualVector, especially when leveraging continuous arguments and self-refinement (see main paper's Table 2 and Table 3). The self-refinement modules yield a notable increase in both continuity and alignment accuracy: e.g., in ablation, alignment accuracy jumps from 0.376 to 0.969 and continuity accuracy from 0.686 to 0.886 with refinement enabled.

Qualitative examples show smoother curve transitions and cleaner axis alignment, directly impacting downstream editability and tool compatibility. Recomposed glyphs from Latin and Chinese datasets show marked improvements in fine-grained geometric regularity.

Figure 6: Comparisons of reconstruction quality across encoding/decoding glyph sets, showing ground truth, DeepVecFont-v2, and DesigNet with self-refinement.

DesigNet demonstrates robust generalization in icon datasets as well, with accurate geometric reconstruction from latent representations.

Ablation and Generalization

Ablation studies validate the additive value of continuous parameterization, hierarchical latent structure, and self-refinement modules. The transition to continuous SVG arguments is especially impactful, and self-refinement becomes essential when discretization-induced alignment is no longer present.

Generalization to Chinese fonts confirms the flexibility of DesigNet's architecture and refinement modules, with similar trends in all metrics.

Implications and Future Work

DesigNet's explicit geometric regularization aligns deep SVG synthesis with professional design workflows, making outputs directly amenable to manipulation in commercial tools. The partitioned latent architecture supports style-content disentanglement and meaningful interpolation, which are essential for creative vector design.

From a theoretical perspective, DesigNet advances the integration of deterministic geometric operators and differentiable programmatic reasoning into generative neural models. The use of straight-through estimators to funnel geometric choices into end-to-end training is specifically impactful, opening potential for similar strategies in domains requiring combinatorial or programmatic post-processing.

However, the absolute coordinate approach limits exact motif reuse across glyphs (e.g., repeated serifs) and constrains model compositionality. Promising directions include incorporating diffusion or flow-matching decoders, explicit programmable motif composition, and extension towards ideographic scripts with compositional assembly.

Conclusion

DesigNet represents a significant advance in neural SVG generation, aligning architectural choices and loss formulation with geometric principles valued by expert designers. By integrating differentiable continuity and alignment refinement into a hierarchical Transformer-VAE, DesigNet produces SVG outputs that are more accurate, editable, and stylistically consistent—propelling vector graphics generation towards direct utility in professional creative workflows.

(2604.06494)