Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching

Abstract: Visual illusions traditionally rely on spatial manipulations such as multi-view consistency. In this work, we introduce Progressive Semantic Illusions, a novel vector sketching task where a single sketch undergoes a dramatic semantic transformation through the sequential addition of strokes. We present Stroke of Surprise, a generative framework that optimizes vector strokes to satisfy distinct semantic interpretations at different drawing stages. The core challenge lies in the "dual-constraint": initial prefix strokes must form a coherent object (e.g., a duck) while simultaneously serving as the structural foundation for a second concept (e.g., a sheep) upon adding delta strokes. To address this, we propose a sequence-aware joint optimization framework driven by a dual-branch Score Distillation Sampling (SDS) mechanism. Unlike sequential approaches that freeze the initial state, our method dynamically adjusts prefix strokes to discover a "common structural subspace" valid for both targets. Furthermore, we introduce a novel Overlay Loss that enforces spatial complementarity, ensuring structural integration rather than occlusion. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art baselines in recognizability and illusion strength, successfully expanding visual anagrams from the spatial to the temporal dimension. Project page: https://stroke-of-surprise.github.io/

• The paper's main contribution is the introduction of a dual-branch optimization pipeline that enables progressive semantic transitions in vector sketches.
• It employs differentiable rasterization and a novel Overlay Loss to maintain non-redundant stroke evolution and dual-target alignment.
• Experimental results and user studies validate superior performance in semantic concealment and structural integration compared to baseline methods.

Progressive Semantic Illusions in Vector Sketching: A Technical Synthesis

Introduction

"Stroke of Surprise: Progressive Semantic Illusions in Vector Sketching" (2602.12280) introduces a new generative paradigm in vector graphics—progressive semantic illusions—where a single vector sketch undergoes dramatic semantic transitions as strokes are added, shifting from one concept to another in a temporally coherent fashion. This work formulates the dual-constraint vector synthesis problem and proposes a sequence-aware joint optimization pipeline grounded in differentiable rasterization and dual-branch Score Distillation Sampling (SDS), enabling both the prefix and cumulative strokes to simultaneously satisfy distinct text-to-visual objectives. A crucial innovation, Overlay Loss, enforces spatial coordination, ensuring that the sketch evolves without semantic or structural redundancy.

Problem Formulation and Methodology

The task requires synthesizing a sketch that is initially recognizable as one object (e.g., a pig) and, after augmenting with additional strokes, transforms into a distinct object (e.g., an angel). Formally, the stroke set $S$ is partitioned into $S_\text{prefix}$ and $S_\text{delta}$: the prefix must depict the initial target $p_1$ and, together with the delta, form the final target $p_2$. The proposal is distinct from prior spatially-manipulated or static illusion synthesis—progression is temporal and cumulative, not achieved through global image editing or viewpoint shifts.

The methodological pipeline iteratively optimizes parameters of differentiable Bézier strokes, employing two main innovations:

• Dual-Branch SDS: Parallel optimization branches independently guide (1) the prefix-only strokes toward $p_1$ and (2) the prefix-plus-delta strokes toward $p_2$, sharing learnable parameters such that the prefix optimally serves dual roles.
• Overlay Loss: To prevent trivial occlusion-based solutions (i.e., overwriting the prefix with delta strokes), a soft spatial overlay penalty is computed using Gaussian-blurred rasterizations, forcing spatial complementarity between subsets.

This is illustrated in the pipeline overview below.

Figure 2: The dual-branch optimization pipeline enables simultaneous guidance from two semantic targets, separating prefix and delta strokes for compounded text-to-sketch alignment and spatial integration.

Overlay Loss and Structural Coordination

The overlay loss addresses a critical failure mode of unconstrained generative pipelines: superfluous or redundant strokes that undermine clarity or semantic distinction between phases. This approach computes a differentiable overlap metric over blurred binary masks for $S_\text{prefix}$ and $S_\text{delta}$, with the penalty proportional to their spatial intersection in latent space. This enforces a “buffer” zone, maintaining the visibility and functional integration of the initial semantics throughout the transformation process.

Figure 1: Overlay loss: soft-blurred maps extend the penalty region, ensuring delta strokes maintain a buffer from the prefix scaffolding, resulting in cleaner, more interpretable phase transitions.

Evaluation Protocols and Ranking

To evaluate both technical and perceptual efficacy, the framework employs a multi-stage filtering and ranking protocol:

• VLM-based Assessment: GPT-4o is leveraged for semantic and structural recognition, evaluating not just prefix and final recognizability, but also the indispensability of prefix strokes in supporting the final phase. Illusion quality is explicitly computed as the differential gain in recognizability attributable to the inclusion of prefix strokes, filtering trivial or degenerate solutions.

  Figure 3: VLM-based ranking pipeline: phase-wise recognizability and semantic scaffolding assessed by GPT-4o, ensuring meaningful phase transitions and non-trivial structural reuse.

• Metric-Based Ranking: A composition of CLIP, ImageReward, and HPS metrics penalizes cases where delta strokes alone are highly recognizable, reflecting the necessity of progressive construction and semantic concealment.

Multi-Phase Extension

The architecture natively scales to $K$-phase illusions, where strokes are partitioned into multiple cumulative subsets (e.g., apple $\rightarrow$ sheep $\rightarrow$ Einstein), each guided by separate text objectives and overlay penalties. This is realized by extending the dual-branch paradigm to $K$ parallel branches, ensuring stroke parameters are optimized with gradients from all subsequent semantic constraints.

Figure 4: Multi-phase joint optimization enables cumulative guidance and overlay constraints, seamlessly chaining semantic illusions across several drawing stages.

Experimental Results

Comprehensive quantitative and qualitative experiments demonstrate significant advantages over vector (SketchDreamer, SketchAgent) and raster baselines (Nano Banana Pro):

• Semantic Concealment and Structural Integration: The proposed method achieves higher CLIP and structural concealment scores, with full coverage and markedly fewer visual artifacts of clutter or destructive editing.

  Figure 5: Qualitative comparison against strong text-to-vector and raster baselines; only the proposed method preserves dual recognizability and structural reuse across progressive transitions.

• User Studies: Empirical studies with 143 participants reveal overwhelming preference for the proposed method's outputs, with satisfaction rates exceeding 97% for joint optimization and ranking pipeline.

Figure 6: User study results demonstrating robust user preference and methodological reliability for progressive illusion synthesis.

Ablation Studies

Three ablations elucidate the necessity of design choices:

• Optimization Strategy: Sequential generation leads to prefix rigidity and semantic conflict, whereas joint optimization (the proposed approach) finds a common structural subspace that underpins smooth, versatile transitions.

  Figure 9: Sequential baselines fail to support dual semantics; joint optimization discovers flexible subspaces for feature reuse.

• Overlay Loss: Omission of overlay loss leads to redundant, cluttered stroke sets and loss of semantic distinction; including the loss enforces cleaned, non-overlapping representations.

  Figure 7: Overlay loss prevents stroke clumping; its presence is essential for clear, integrated phase transitions.

• Stroke Initialization: Center-gathered initialization balances semantic density and spatial coverage, critical for non-convex optimization problems.

  Figure 8: Initialization impacts optimization convergence and structural validity; the centered gathering outperforms alternatives.

Applications and Generalization

The framework generalizes across concept complexity (e.g., simple to highly detailed subjects), number of phases, and vector representations (e.g., B-splines, colored strokes). Extended benchmarks further establish its robustness for temporal visual anagram synthesis.

Implications and Future Directions

This work expands the generative sketching paradigm from static and spatially-constrained illusions to temporally-cumulative, semantically progressive syntheses. The dual-constraint, overlay-enforced pipeline is broadly applicable to animation, co-creative human-AI illustration, and educational visualization tools. Prospective research includes further integration with LLM-based sketch planning, stronger priors for complex structural concepts, and human-in-the-loop editing for controlled semantic reversibility.

Conclusion

"Stroke of Surprise" introduces a technically rigorous and extensible pipeline for generating progressive semantic illusions in vector sketches. The dual-branch SDS and overlay loss address the combinatorial and geometric challenges inherent in dual-constraint optimization. Empirical and user-based metrics—along with ablative evidence—affirm that joint parameter sharing and non-redundant structural design are essential for high-quality progressive illusions. The methodology opens new trajectories for temporally-evolving, semantically-rich sketch generation.