OptiClear: Differentiable Curvilinear Design Rule Legalization for Inverse-Designed Photonic Devices

Abstract: Photonic inverse design enables ultra-compact, high-performance devices with highly curvilinear and non-intuitive geometries, but the resulting layouts often violate fabrication design rules and hinder foundry manufacturing. Legalization methods designed for rectilinear Manhattan electrical layouts are not directly applicable to curvilinear inverse-designed photonic devices. Meanwhile, existing fabrication-aware inverse-design methods apply soft penalties on small features and sharp curvatures, but still cannot guarantee design-rule-compliant final layouts. In this work, we present OptiClear, a curvilinear design rule legalization framework for inverse-designed photonic devices. OptiClear provides two complementary legalization engines: OptiClear-R, a rule-based morphological legalizer that efficiently resolves violation regions through iterative morphology-guided mask processing, and OptiClear-D, a differentiable legalizer that formulates legalization as a minimum-distortion mask optimization problem under morphological stationary-point constraints, explicitly seeking a rule-compliant layout with minimal geometric deviation from the original design. We further develop customized differentiable morphological GPU operators that significantly improve the scalability of high-resolution mask legalization. Comprehensive evaluation across diverse inverse-designed photonic devices and a wide range of design-rule settings shows that OptiClear reduces design-rule violations from thousands to zero. The rule-based legalizer offers high runtime efficiency, while the differentiable legalizer more faithfully preserves the original optical functionality. This work establishes curvilinear design rule legalization as a practical post-design electronic-photonic design automation (EPDA) stage for translating high-performance inverse-designed photonic layouts into manufacturable tape-out-ready devices.

• The paper introduces a dual-engine framework, OptiClear-R and OptiClear-D, to ensure manufacturability of inverse-designed, curvilinear photonic layouts.
• The paper leverages morphological image operations and a differentiable optimization approach to enforce complex fabrication rules while minimizing geometric distortion.
• The paper demonstrates significant runtime gains and minimal functional loss, achieving zero design rule violations for foundry-ready photonic device masks.

OptiClear: Differentiable Curvilinear Design Rule Legalization for Inverse-Designed Photonic Devices

Introduction

The transition from high-performance inverse-designed photonic devices to manufacturable layouts remains a foundational bottleneck in photonic integrated circuit (PIC) development. Inverse design produces highly non-intuitive and curvilinear geometries engineered for exceptional functionality but typically rife with design rule violations (DRVs), making them incompatible with foundry fabrication processes. Conventional electronic legalizers targeting rectilinear layouts and soft regularization mechanisms incorporated in fabrication-aware inverse design (FAID) frameworks are insufficient for the explicit enforcement of photonic geometry constraints.

This work introduces OptiClear, a formal, post-design curvilinear legalization framework comprised of two complementary engines—OptiClear-R (rule-based morphological legalizer) and OptiClear-D (differentiable, minimum-distortion legalizer)—to ensure manufacturability of arbitrary curvilinear photonic layouts. OptiClear is shown to reliably eliminate all true DRVs across a wide variety of high-performance designs, offering distinct trade-offs in runtime efficiency and geometry/functionality preservation.

Figure 1: Example inverse-designed photonic device with representative fabrication design rule violations, which hinder the practical deployment of ID devices in manufacturable PICs.

Photonic Curvilinear Design Rules and Morphology

Conventional EDA legalization techniques presuppose rectilinear, Manhattan geometry, and are inapplicable to the arbitration of DRVs in inverse-designed PICs, which instead display dense, subwavelength, curvilinear details and irregular topologies. In these masks, minimum width, spacing, area, curvature, and similar constraints are coupled and often conflicting, so local edits frequently induce new violations.

Morphological image operations, specifically opening and closing, provide a natural vocabulary for DRV removal; opening eliminates small foreground features and thin bridges, while closing fills small holes and narrow gaps. However, naïve alternation between these operators can lead to oscillatory behavior in the presence of "conflict zones"—geometries where enforcement of spacing and width rules compete.

Figure 2: Morphological Open and Close operations.

Morphological Kernel Size Derivation

Morphological legalization success depends on processing in the original pixelated mask domain and selecting a kernel size aligned with design rule requirements. The morphological kernel must guarantee enforcement of minimum width, minimum spacing, and implicitly, curvature rules, accounting for discretization-induced error in pixel-polygon conversion. The kernel size is analytically derived as a function of edge-to-edge distance, local polygon angle $\theta$, and a safety margin to guarantee robustness to polygon simplification.

Figure 3: Illustration of design rule check for curvilinear photonic device masks and the derivation of the morphological kernel size. Morphological kernel size is determined by the edge-to-edge distance rule, curvature requirements, and error margin.

OptiClear-R: Rule-Based Morphological Legalizer

OptiClear-R systematically resolves DRVs by breaking conflict zones and projecting the mask towards a morphological stationary point, where the mask is invariant under both opening and closing. The procedure prioritizes silicon region preservation—minimizing core perturbation that would degrade device performance—using skeletonization and systematic dilation of foreground or background depending on region.

A conflict arises when opening tends to erode thin silicon and closing tends to reconnect features just separated, resulting in non-convergent cycles. OptiClear-R instead applies preprocessing with smaller kernels, targeted iterative skeleton-based dilation, and halts upon open/close equilibrium.

Figure 4: Conflict zone between morphological opening and closing. Opening removes narrow connections while closing tends to reconnect features; their effects can be inconsistent in the absence of equilibrium.

Figure 5: Flow of the proposed rule-based legalizer OptiClear-R. Foreground dilation is prioritized to preserve silicon connectivity until mask stability is achieved.

OptiClear-D: Differentiable Minimum-Distortion Legalization

Recognizing the sensitivity of device performance to geometric distortion, OptiClear-D casts legalization as a constrained, differentiable optimization over a level-set-parametrized mask. The objective minimizes the $L_2$ deviation from the original mask, under the constraint that the solution is a stationary point of both opening and closing with the derived kernel.

This constrained minimization is solved via an augmented Lagrangian approach, with pixel-wise Lagrange multipliers enforcing the morphological stationary condition. Notably, OptiClear-D introduces custom Triton-accelerated differentiable morphological operators, achieving massive scaling on large, high-resolution masks (e.g., 4096$\times$4096 pixels), far outperforming prior dense implementations in both speed and memory consumption.

Figure 6: Flow of the proposed differentiable legalizer OptiClear-D. The mask is optimized in the latent level-set space initialized from the original mask.

Figure 7: Solution space illustration and optimization dynamics of OptiClear-D.

Figure 8: OptiClear’s Triton-based kernel inference achieves orders-of-magnitude speedup for large-scale, high-resolution masks without OOM issues.

Quantitative Evaluation

Comprehensive empirical validation demonstrates that OptiClear-R and OptiClear-D eliminate all true DRVs (zero after filtering for algorithmic and DRC tool false-positives) across diverse photonic components: waveguide bends, crossings, circulators, splitters, multiplexers, etc. All masks are manufactured with aggressive minimum-feature-size (MFS) settings and design rules.

Figure 9: Benchmark examples of inverse-designed photonic devices; after OptiClear-D, all true DRVs are eliminated.

Other highlights include:

• OptiClear-D achieves superior optical performance preservation versus rule-based, open/close, or soft-penalty FAID baselines. This is especially manifest in performance-sensitive regions (i.e., near device ports), where direct gap/curvature penalty or rule-based fixes fail to remove critical DRVs without excessive geometry displacement.
• The geometry–functionality correlation is weak: global similarity ($L_2$, IoU) does not guarantee high post-legalization figure-of-merit (FoM), emphasizing the significance of distortion-minimized, constraint-aware mask optimization.
• OptiClear-R provides an order-of-magnitude runtime gain compared to OptiClear-D, making it suitable for rapid, less-sensitive jobs or as a pre-filtering stage.

  Figure 10: A spacing violation in a performance-sensitive region that cannot be removed by FAID penalties; OptiClear-D introduces minimal displacement, whereas OptiClear-R merges silicon to enforce legality.

Adaptive studies (on varying feature size, DRV rule stringency, and mask resolution) show that:

• Strong MFS control during inverse design reduces the DRV correction burden but cannot obviate the need for explicit legalization.
• As DRV stringency increases relative to MFS, post-legalization FoM degrades, and the need for joint design–legalization co-optimization becomes evident.
• Full (1 nm/pixel) mask resolution is required for DRV-clean post-legalization; downsampling leads to irreparable violations.

  Figure 11: The MSE loss in OptiClear-D suppresses mask distortion, resulting in better function preservation during legalization.

Practical and Theoretical Implications

OptiClear provides the first scalable, differentiable, and rule-guaranteed legalization solution for curvilinear photonic layout, forming a critical post-design stage in any realistic electronic-photonic design automation (EPDA) flow. From a practical standpoint, adoption allows high-throughput translation of arbitrary inverse-designed layouts to foundry-ready tape-outs without manual intervention.

Theoretically, the explicit formulation of mask legalization as a morphology-constrained, distortion-minimizing problem, executable via differentiable and GPU-accelerated operators, bridges the gap between categorical, hard-rule enforcement and soft, penalty-driven optimization. This also enables, for the first time, end-to-end differentiable photonic design flows that can backpropagate design rule constraints into generative neural architecture or adjoint-based topology optimization.

Conclusion

OptiClear establishes post-design curvilinear DRV legalization as an essential and technically tractable element of manufacturable photonic device design. With co-optimized rule-based and differentiable engines, scalable GPU morphology, and formal constraint satisfaction, OptiClear solves the hitherto open challenge in electronic-photonic design: enabling zero-DRV tape-out of arbitrary, high-performance inverse-designed device layouts with minimal functional loss. The formulation and morphologically differentiable backpropagation also open new directions for integrating rule-awareness directly into future end-to-end AI-augmented photonic design pipelines.