Differentiable Stroke Planning with Dual Parameterization for Efficient and High-Fidelity Painting Creation

Abstract: In stroke-based rendering, search methods often get trapped in local minima due to discrete stroke placement, while differentiable optimizers lack structural awareness and produce unstructured layouts. To bridge this gap, we propose a dual representation that couples discrete polylines with continuous Bézier control points via a bidirectional mapping mechanism. This enables collaborative optimization: local gradients refine global stroke structures, while content-aware stroke proposals help escape poor local optima. Our representation further supports Gaussian-splatting-inspired initialization, enabling highly parallel stroke optimization across the image. Experiments show that our approach reduces the number of strokes by 30-50%, achieves more structurally coherent layouts, and improves reconstruction quality, while cutting optimization time by 30-40% compared to existing differentiable vectorization methods.

• The paper introduces a novel differentiable stroke planning framework employing dual parameterization to enhance both reconstruction quality and editability.
• It utilizes a two-stage method—greedy polyline search followed by adaptive Bézier conversion—achieving SSIM of 0.93, PSNR of 32.16 dB, and a 52.2× speedup over prior methods.
• The approach offers robust editability and precise artistic control through physically-based rendering techniques that simulate both 2D and 3D effects.

Differentiable Stroke Planning with Dual Parameterization for Efficient and High-Fidelity Painting Creation

Overview

The paper introduces a novel differentiable stroke planning (DSP) framework that enhances the reconstruction quality, efficiency, and editability of stroke-based rendering (SBR) systems. Central to the approach is a dual-parameterization scheme for representing strokes, leveraging both polylines and piecewise cubic Bézier curves. This system supports adaptive stroke complexity and robust optimization, facilitating superior reconstruction fidelity and controllable stylistic abstraction.

Methodology

The DSP method consists of a two-stage strategy: greedy polyline search followed by adaptive conversion to piecewise cubic Bézier curves. The polyline stage exploits residual gradients to iteratively position vertices, with termination based on step count or gradient saturation. The subsequent Bézier conversion balances expressiveness and stability, with the number of control points dynamically adjusted according to trajectory complexity detected in the search phase.

Direction regularization ensures polyline smoothness, mitigating topological degeneracy during curve fitting. Rendering utilizes an analytic kernel and physically-inspired, anisotropic Gaussian splatting to produce both realistic and painterly effects. This dual parameterization not only increases fidelity but also allows for consistent, user-controllable trade-offs between abstraction and realism.

Efficiency and Fidelity Analysis

Strong numerical results underscore the method's performance relative to state-of-the-art (SOTA) baselines. On the DIV2K validation set, the proposed method surpasses others in both perceptual and geometric fidelity. Notably, compared to SBP and ALPA, it yields an SSIM of 0.93 (+0.16 over ALPA), PSNR of 32.16 dB, and a substantial reduction in LPIPS (0.076). Edge consistency is also superior at 0.91. The reconstruction pipeline demonstrates significant acceleration—rendering and optimization on 2K images is completed with a 52.2× speedup over DiffVG, while maintaining higher output PSNR. These improvements are due to the algorithm's analytic splatting, Bézier fitting, and effective stroke reuse via reinitialization.

Editability and Artistic Control

The dual-stroke representation enables robust editability and nuanced stylistic control. Artistic variance is managed via search extent and the rendering kernel's hardness parameter $\tau$: low $\tau$ values yield sharper, brush-like strokes, while higher values provide softer, photorealistic blending. All stylistic settings preserve structural fidelity and painterly coherence, further empowering artists and SBR applications demanding high customizability.

The approach's physical rendering model incorporates volumetric stroke height fields, permitting physically-plausible 3D effects and laying a foundation for future integration with 3D printing or advanced painterly simulation pipelines.

Figure 1: Visualization of the painting process, illustrating the transition from coarse abstraction to fine-grained detail as the stroke count increases.

Comparison with Related Work

In contrast to standard vectorization—often producing inflexible, closed tessellations—the method's skeleton-based, open stroke representations better capture the intent and process of human painters. Compared with differentiable vector graphics like DiffVG, the presented DSP scheme excels in efficiency, realism, and support for physically-motivated attributes.

For SBR tasks requiring fidelity to both edge and texture statistics, the method demonstrates advances over SOTA in both metrics and rendering runtime. This generalized framework also supports downstream tasks, including reconstruction of artistic paintings, photo-to-painting translation, and interactive vector graphic editing.

Practical and Theoretical Implications

Practically, these advances make high-fidelity, editable painting synthesis feasible for large-scale creative and industrial applications, such as digital art tools and rendering engines. Theoretically, the adaptive, differentiable dual parameterization strengthens the bridge between classic raster-image optimization and modern differentiable graphics, providing new pathways for integration with neural SBR and inverse graphics models.

This architecture also opens venues for further research on differentiable forward models of painting, neural trajectory prediction, and the optimization of aesthetic metrics directly within the DSP loop. The explicit incorporation of physically-based 3D representations further enhances interactivity and realism in digital artist workflows.

Future Directions

Potential extensions include the unification of DSP with neural SBR modules, exploration of multimodal style transfer within the dual-parameterization regime, and leveraging height field outputs for tangible printing. Future research might focus on real-time DSP for interactive applications, broader stylistic domains, and incorporation of richer physical brush and medium models.

Conclusion

The dual-parameterized DSP framework substantially elevates the standard for fidelity, efficiency, and editability in painting creation. By aligning stroke search, representation, and rendering in a fully differentiable pipeline, it addresses key limitations of prior SBR methods. The system's strong empirical results and extensible foundation make it a compelling contribution to both differentiable graphics and digital painting.