Illustrator’s Depth

• Illustrator’s Depth is a computational approach for extracting and representing non-metric depth in digital illustrations through layer ordering and occlusion cues.
• It employs techniques like shape layer graph construction and iterative layer extraction to reconstruct and manipulate layered visuals.
• Applications include vectorization, re-editable layer decomposition, and 3D asset synthesis, bridging creative design and geometry-aware processing.

The concept of "Illustrator’s Depth" encompasses the computational extraction, representation, and manipulation of visual depth in digital illustrations, vector graphics, and stylized images. Unlike natural photographs, illustrative content such as comics, cartoons, and graphic designs often encode depth solely through occlusion, form, and artistically constructed layering, lacking explicit 3D cues like shading or stereo disparity. Recent research addresses the challenge of inferring and utilizing such depth for applications in vectorization, re-editable layer decomposition, 3D asset synthesis, and semantically grounded generative modeling.

1. Formalization of Depth in Illustration

In digital illustration domains, "depth" predominantly arises from explicit or implicit z-ordering of layers, figure-ground segregation, and occlusive relationships among planar shapes. Unlike photometric depth in natural images, illustrative depth is non-metric and closely aligned to the rendering pipeline of design software (e.g., Illustrator, Photoshop) where scene elements are assigned to layers that are composited in a defined stacking order.

Mathematically, this compositing can be modeled by recursively applying the over-operator to a stack of RGBA layers $\{l_k\}_{k=0}^K$, where the $k$-th composited image $x_k^C$ is

$$x_k^C = l_k^C \odot l_k^A + x_{k-1}^C \odot (1 - l_k^A),$$

with $l_k^C$ the RGB and $l_k^A$ the alpha channel of layer $k$ (Suzuki et al., 29 Sep 2025). Accurate reconstruction of this depth ordering from a flattened raster image is fundamentally ill-posed due to loss of information during compositing.

2. Algorithms for Inferring Depth and Layering

A. Shape Layer Graph Construction

Recent models segment a color-quantized input image into shape layers by connected-component analysis in color space (Law et al., 10 Sep 2024). Each layer $S_i$ is defined as a maximal region of uniform color:

$$S_i \subset \Omega,\quad f(x) = c_{\ell_i}\ \forall x \in S_i,$$

where $f$ is the quantized image. These are nodes in a directed graph $G=(V,E)$ encoding front-back (occlusion) relations.

B. Depth Ordering Energy

To infer relative stacking, pairwise occlusion is measured using the fraction of area of one shape under the convex hull of another:

$$A(i, j) = \frac{\int_\Omega \chi_i(x)\, \chi_j^{\mathrm{Conv}}(x)\,dx}{\int_\Omega \chi_i(x)\,dx},$$

yielding an antisymmetric score $D(i, j) = A(i, j) - A(j, i)$. By thresholding $D(i, j)$, directed edges $i \rightarrow j$ (front-to-back) are assigned. Cycles, if present in $G$, are resolved via minimizing a convex hull symmetric difference metric $V(i, j)$, iteratively deleting the edge with highest $V$ until acyclicity is achieved (Law et al., 10 Sep 2024).

C. Iterative Layer Extraction

LayerD approaches layer restoration from raster graphics via a feed-forward matting-inpainting loop. For each iteration:

• Matting network $F_\theta$ estimates the visible alpha mask $\hat\alpha_m$ of the topmost layer.
• Inpainting network $G_\phi$ fills occluded regions, yielding background $\hat x_{m-1}$.
• The top-layer’s RGB is "unblended":

$$\hat l_m^C = \frac{\hat x_m^C - \hat x_{m-1}^C \odot (1-\hat \alpha_m)}{\hat \alpha_m},$$

progressively peeling off layers until no visible foreground remains (Suzuki et al., 29 Sep 2025).

D. Curvature-Based Convexification

For inferring hidden (occluded) regions of partial shapes, convexification is performed by variational inpainting minimizing the Euler’s elastica over candidate regions:

$E(C_i) = \int_{\partial C_i} (a + b\kappa^2) ds,$

with $\kappa$ the curvature, subject to Dirichlet boundary constraints. Modica–Mortola phase-field approximation enables efficient minimization and extension of visible shape boundaries based on perceptual smoothness priors (Law et al., 10 Sep 2024).

3. Applications: Vectorization, Decomposition, and Generation

A. Vectorization with Depth Ordering

Law & Kang’s model outputs vector graphics (SVG) where each shape’s boundary is fit as a sequence of cubic Bézier curves, and the computed topological sort determines their z-order in the output. Semantic grouping post-processes color-quantized segments to combine over-split regions using unsupervised segmentation energies (Law et al., 10 Sep 2024).

B. Re-editable Layer Decomposition

LayerD recovers RGBA layers from a rasterized composite, enabling re-editable workflows analogous to native vector graphics. Uniform-appearance priors (region-wise palette quantization) refine extracted alpha masks and colors, improving downstream applications like style transfer, object rearrangement, and editing (Suzuki et al., 29 Sep 2025).

C. 3D Asset Synthesis from Flat Illustrations

Art3D lifts flat-colored 2D designs into plausible 3D, pipeline stages:

• Structure proxy augmentation: proxies simulating depth and edge cues via MiDaS and Canny, used as condition inputs for ControlNet variants,
• Visual-LLM (VLM) selection: ranking outputs for strongest 3D illusion,
• Pretrained 3D generator (Trellis) for shape, and pretrained diffusion-based texturing for style restoration. No explicit loss functions for geometric consistency are used; instead, pipeline selection and priors are crucial. The method generalizes across styles and benchmarks with qualitative user studies (Flat-2D dataset) (Cong et al., 14 Apr 2025).

D. Depth-Aware Composable Synthesis

Compose and Conquer (CnC) introduces depth disentanglement in diffusion models for compositional generative tasks. Separate depth maps for the foreground and background, derived from masked monocular estimates (e.g., MiDaS), are fused into a U-Net backbone. Soft guidance via cross-attention masking prevents semantic bleed while enabling region-specific global style control. CnC allows explicit placement and occlusion ordering in the generated image; evaluated on COCO-Stuff and Pick-a-Pic with best-in-class FID, IS, CLIPScore, SSIM, LPIPS, Depth MAE (Lee et al., 17 Jan 2024).

4. Dataset Construction and Evaluation Metrics

• Layered Datasets: LayerD evaluates on Crello composites with synthetic and real designs. Ground-truth layers, if present, are used to assess reconstruction fidelity.
• Edit-Aware Metrics: Dynamic Time Warping (DTW) aligns predicted and GT layers, employing weighted RGB error and alpha-IoU:

$$d(k,q) = w_C \|\hat l_k^C - l_q^C\|_{1,\alpha} + w_A (1-\mathrm{IoU}(\hat l_k^A, l_q^A)).$$

Layer merge operations accommodate differences in annotation granularity; lower DTW cost reflects reduced user edit effort (Suzuki et al., 29 Sep 2025).

• Depth Metrics for Illustration: Comics-depth models use human-annotated per-pixel ordinal depth, reporting AbsRel, SqRel, RMSE, RMSE(log) over DCM, eBDtheque datasets (Bhattacharjee et al., 2021).
• 3D Generation Assessment: Art3D forgoes numeric metrics in favor of qualitative visual fidelity, style preservation, and user preference studies (Cong et al., 14 Apr 2025).

5. Limitations and Practical Considerations

• Ambiguity in Monocular Depth: Illustrative depth is inherently ambiguous without explicit 3D cues. Extracted layering or z-order may diverge from artistic intent, particularly in stylized or abstract content (Law et al., 10 Sep 2024, Cong et al., 14 Apr 2025).
• Reliance on Pretrained Priors: Most methods depend on pretrained diffusion, VLMs, or depth estimation backbones with their built-in biases; transfer to unseen illustration styles may require domain adaptation (Cong et al., 14 Apr 2025, Bhattacharjee et al., 2021).
• Evaluation Challenges: Lack of reliable ground-truth layers (due to designer variability and annotation granularity) motivates use of edit-aware metrics and human-in-the-loop assessment (Suzuki et al., 29 Sep 2025).
• Optimization-Free Pipelines: Recent systems such as Art3D and LayerD run in zero-shot, optimization-free fashion, relying on forward inference and deterministic refinement heuristics rather than explicit loss minimization (Cong et al., 14 Apr 2025, Suzuki et al., 29 Sep 2025).

6. Interdisciplinary and Future Directions

"Illustrator’s Depth" provides structural insights critical for bridging the gap between 2D artistic content and geometry-aware computation:

• Semantic 3D Scene Understanding: Leveraging inferred illustrator-style depth for high-level reasoning, object relationship extraction, and spatial composition.
• Hybrid Layering and Geometry: Integrating vectorization pipelines with 3D lifting (e.g., convexified SVG layers mapped to 3D surfaces) to support seamless 2D/3D creative workflows.
• Generalization Beyond Illustration: Extending current monocular-depth and layer-inference approaches to broader stylized domains (e.g., scientific diagrams, manga, education graphics) with style-adaptive priors and more robust occlusion reasoning.
• Unified Representational Frameworks: Deepening the mathematical and computational integration of depth, layering, vectorization, and volumetric synthesis, yielding tools that respect both artistic intention and analytic tractability (Lee et al., 17 Jan 2024, Law et al., 10 Sep 2024, Bhattacharjee et al., 2021).

Recent progress in illustrator's depth has advanced the ability to reconstruct, manipulate, and augment depth for a broad class of non-photorealistic visuals, with applications spanning graphics, augmented reality, generative design, and re-editable digital media.