AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling

Abstract: We introduce AmodalSVG, a new framework for amodal image vectorization that produces semantically organized and geometrically complete SVG representations from natural images. Existing vectorization methods operate under a modal paradigm: tracing only visible pixels and disregarding occlusion. Consequently, the resulting SVGs are semantically entangled and geometrically incomplete, limiting SVG's structural editability. In contrast, AmodalSVG reconstructs full object geometries, including occluded regions, into independent, editable vector layers. To achieve this, AmodalSVG reformulates image vectorization as a two-stage framework, performing semantic decoupling and completion in the raster domain to produce amodally complete semantic layers, which are then independently vectorized. In the first stage, we introduce Semantic Layer Peeling (SLP), a VLM-guided strategy that progressively decomposes an image into semantically coherent layers. By hybrid inpainting, SLP recovers complete object appearances under occlusions, enabling explicit semantic decoupling. To vectorize these layers efficiently, we propose Adaptive Layered Vectorization (ALV), which dynamically modulates the primitive budget via an error-budget-driven adjustment mechanism. Extensive experiments demonstrate that AmodalSVG significantly outperforms prior methods in visual fidelity. Moreover, the resulting amodal layers enable object-level editing directly in the vector domain, capabilities not supported by existing vectorization approaches. Code will be released upon acceptance.

• The paper introduces a novel two-stage process that achieves semantically coherent and amodally complete SVGs by combining semantic layer peeling with adaptive layered vectorization.
• The methodology employs VLM-guided iterative decomposition and hybrid inpainting to ensure geometric closure and enhanced object-level editing.
• Empirical results demonstrate significant improvements in PSNR, LPIPS, and SSIM, setting a new standard for editable vector graphics in complex scenes.

AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling

Introduction and Motivation

AmodalSVG presents a paradigm shift in image vectorization by directly addressing one of the intrinsic limitations of existing vectorization frameworks—their inability to semantically disentangle and amodally complete object representations in vector graphics. Traditional approaches, including differentiable rendering and path-by-path or coarse-to-fine strategies, inherently depend on a modal paradigm that only considers visible pixel regions. Consequently, generated SVGs lack both semantic modularity and geometric completeness, precluding meaningful object-level editing such as manipulation, reordering, or removal without artifact generation. AmodalSVG, in contrast, reformulates vectorization as a two-stage process that first performs semantic layer decoupling and amodal completion in the raster domain before layer-wise vectorization, thereby bridging the gap between perception-driven scene parsing and vector-based compositional graphics.

Figure 1: AmodalSVG vectorizes natural images into semantically decoupled, amodally complete SVG layers, enabling object-level editing (e.g., recolor, reorder, resize, remove) with preserved visual consistency.

Methodology

Semantic Layer Peeling (SLP)

Central to the pipeline is Semantic Layer Peeling (SLP), an iterative VLM-guided decomposition that identifies and extracts the topmost semantic entity at each recursion, followed by holistic amodal inpainting of both foreground and background regions. Advanced VLMs (e.g., Qwen3-VL) are prompted to perform explicit depth and occlusion reasoning, generating a discriminative semantic tuple at each iteration. Fine-grained semantic grounding is achieved by feeding VLM-discriminated keywords into GroundedSAM for pixel-accurate mask extraction.

Foreground completion utilizes dynamic canvas padding for boundary-truncated objects to guarantee geometric closure, while background restoration harnesses a hybrid inpainting pipeline (cascaded LaMa and FLUX), leveraging both structural continuity and semantically accurate generative synthesis. This hybrid strategy offers advantages over reliance on a single inpainting source, enabling the plausible restoration of background context after each semantic peel, which is substantiated in additional ablations.

Figure 2: Overview of AmodalSVG: SLP recursively extracts semantic layers (left), then ALV vectorizes each layer with adaptive primitive management (right).

Adaptive Layered Vectorization (ALV)

For vectorization, AmodalSVG introduces the Adaptive Layered Vectorization (ALV) mechanism. Departing from fixed heuristic or schedule-based approaches, ALV applies a feedback-driven process: primitives are initially allocated based on the spatial mask coverage and direct sampling from raster colors and positions. Redundant primitives—those whose effective contribution to the observable image after global alpha compositing falls below a learned threshold—are pruned using an occlusion-aware proxy that efficiently approximates each primitive’s visual footprint in the rendered hierarchy.

Refinement is achieved by importance-driven addition, wherein new primitives are spatially sampled from gradients and residual error fields per temperature-scaled softmax. Critically, the required number of new primitives per refinement is predicted via historical marginal loss reduction (error budget), enforcing a tight fidelity-efficiency tradeoff throughout optimization. The total objective function combines pixel-wise reconstruction and alpha-mask boundary losses to promote spatial regularity and prevent path bleeding.

Figure 3: AmodalSVG achieves superior SVG reconstruction fidelity versus prior art, preserving fine details and edge sharpness.

Experimental Results

Quantitative Comparisons

Extensive evaluation on real-world images demonstrates that AmodalSVG surpasses leading baselines across all fidelity metrics. PSNR reaches 32.91 dB, outperforming DiffVG (30.52 dB), with corresponding improvements in LPIPS (0.0554 vs. 0.1075) and SSIM (0.9342 vs. 0.8860). Notably, prior approaches relying on path-by-path or modal region segmentation either generate fragmented, incoherent layers (path-based) or semantically entangled, incomplete masks (coarse-to-fine), as further evidenced by visual quality breakdowns.

Layering Quality and Ablation Analysis

Visual examinations highlight AmodalSVG’s ability to generate semantically and structurally coherent layers, e.g., retrieving full occluded limbs or tails, and ensuring backgrounds are perfectly inpainted even in the presence of nested occlusions. Layers are individually editable, and objects can be removed, recolored, or reordered without background holes—contrast this with baseline methods that either produce persistent artifacts or lose object-level editability by flattening entities into mutually entangled primitives.

Ablation studies establish that both VLM guidance and the iterative decomposition of SLP are critical. Removing VLMs causes severe over-segmentation, while a one-shot VLM approach suffers from missed objects and ambiguity in occlusion boundaries. Likewise, in ALV, content-aware initialization and occlusion-driven pruning are essential for efficient convergence and high-frequency detail recovery, with importance-driven addition achieving the optimal trade-off between primitive complexity and visual quality.

Figure 4: AmodalSVG layering quality: layers are semantically coherent and amodally complete; path-based or pixel-wise methods produce entangled or empty layers.

Figure 5: Ablation of SLP: without VLM or iterative peeling, segmentation is over-fragmented or under-localized. The proposed method achieves robust entity extraction and background completeness.

Editability and Applications

Structurally, AmodalSVG-generated SVGs enable a suite of editing operations (reposition, recolor, replace, remove, resize, reorder) not supported by prior methods. Object manipulation, both in-layer and cross-layer, maintains visual and compositional consistency due to amodal completion and precise semantic isolation. The representation greatly enhances applicability for professional graphic design, animation, and content authoring, opening avenues for downstream applications such as automated UI/UX prototyping, iterative creative workflows, and advanced interactive graphics editing.

Figure 6: AmodalSVG enables seamless object-level manipulation (recolor, move, remove) and robust downstream editing not feasible with previous SVG pipelines.

Limitations and Future Directions

AmodalSVG’s dependence on the capabilities of upstream VLMs introduces failure modes when object parsing or secondary effect interpretation (e.g., transparency, reflection, cyclic occlusions) transcends VLM discriminative capacity. Transmissive entities (e.g., glass) and non-planar occlusion cycles expose topological and photometric assumptions in both SLP and the SVG format itself; these manifest as color blending artifacts or loss of compositional fidelity.

Figure 7: Failure cases: transmissive objects result in unnatural color flattening; cyclic occlusions cannot be rendered due to strictly ordered SVG composition.

Iterative peeling also comes with a direct computational overhead. Future directions proposed include the acceleration of scene decomposition, robustifying VLM-Vision interfaces for complex layer parsing, extending formalism to handle non-strict layer hierarchies (e.g., graph-structured or scene graphs), and incorporating physical effects such as global lighting and shadows natively into the SVG vectorization process.

Conclusion

AmodalSVG establishes a formal, practical methodology for amodal scene vectorization, pairing VLM-based semantic reasoning with raster restoration and feedback-driven geometric vectorization. The framework marks a transition from low-level modal parsing to semantically structured, amodally complete SVGs—substantially expanding the expressive and editing capacity of vector graphics in general-purpose scene synthesis scenarios. Its approach, supported by empirical fidelity gains and transformative editing capabilities, positions it as a reference architecture for future amodal vectorization research and the next generation of editable graphics pipelines.

Reference: "AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling" (2604.10940)