A General Differentiable Ray-Wave Framework for Hybrid Refractive-Diffractive System Modeling and Optimization

Abstract: Hybrid optical systems combining refractive and diffractive optical responses have the potential to support new types of optical behavior, but they are difficult to model and optimize due to the disparate spatial scales and physics exhibited by ray and wave phenomena. In this work, we present a differentiable ray-wave framework that serves as a general model for hybrid refractive-diffractive optical systems and that operates as a plug-and-play module within standard ray tracing pipelines. Our model uniquely applies to both planar and curvilinear diffractive surfaces and can accommodate arbitrary holographic diffractive profiles with high spatial frequency responses. We analyze ray-wave modeling regimes that optimally account for the spatial frequency properties and spatial curvature of the diffractive surfaces, and we demonstrate the gradient-based end-to-end optimization of hybrid refractive-diffractive systems featuring planar and conformal diffractive surfaces. We anticipate that these modeling capabilities will enable new classes of hybrid optical systems relevant to computational imaging and display applications.

• The paper presents a differentiable simulation framework combining local wave-optics and global ray tracing to optimize hybrid refractive-diffractive systems.
• It demonstrates accurate performance with metrics like NCC ≥ 0.99 and improved reconstruction quality through gradient-based inverse design.
• The approach enables modeling of arbitrary DOE profiles and curvilinear geometries, effectively bridging geometric and wave optics for practical device design.

Differentiable Ray-Wave Modeling for Hybrid Refractive-Diffractive Optics

Introduction and Motivation

Hybrid optical systems that integrate refractive elements with complex diffractive optical elements (DOEs), especially those with arbitrary holographic and curvilinear profiles, provide unique opportunities to engineer novel optical behaviors for imaging, display, and non-imaging applications. The interplay between geometric ray optics and diffractive wave optics in such systems is essential for supporting functionalities such as ultra-wide FOV, super-achromatic response, arbitrary wavefront shaping, and conformal configurations. However, accurate and generalizable modeling and optimization of these systems are nontrivial due to the coexistence of fundamentally distinct physics—broadband, field-dependent ray propagation and high-spatial-frequency, coherence-sensitive diffractive responses.

Existing simulation frameworks predominantly suffer from three limitations: (1) applicability is restricted to either smooth DOE phase gradients or planar surfaces, (2) lack of differentiability prevents gradient-based inverse design, and (3) non-generalizability to arbitrary high-frequency, complex-amplitude modulation or curvilinear geometries. The presented work introduces a differentiable ray-wave framework that systematically addresses these limitations for generic hybrid refractive-diffractive optical systems, including those with spatially conformal, high-resolution holographic elements (2605.15418).

Framework Methodology

The core contribution is a modular simulation-optimization strategy combining local wave-optics simulation (via the angular spectrum method, ASM) at DOE-ray interaction sites with global ray tracing for efficient forward and backward propagation:

• Plane Wavelet Representation and Local Patch Modeling: Each geometric ray is modeled as a plane wavelet characterized by amplitude and phase. Upon intersection with a DOE, a local DOE patch (parameterized by user-specified dimensions) is extracted in a local tangent frame.
• Wave Response via ASM and Secondary Ray Sampling: The local patch’s complex amplitude profile is transformed into its angular (wavevector) spectrum using ASM. The resulting distribution is then sparsely converted into a finite, weighted ensemble of secondary rays through Monte Carlo sampling.
• Differentiability through Reparameterization: To maintain gradient flow despite the stochastic sampling of secondary rays, the method employs a Gumbel-Softmax-based reparameterization trick. This enables end-to-end backpropagation required for inverse optics optimization.
• Reconstruction and Coherent Superposition: The global sensor field is coherently reconstructed from all secondary rays using their accumulated optical path and direction, generalizing standard ray-tracing and PSF-based reconstruction approaches.

This framework is agnostic to the surface form (planar or arbitrarily conformal), DOE profile (amplitude, phase, or complex), and the spatial frequency content, thus extending beyond the regime of the generalized law of refraction and enabling simulation for arbitrary DOE configurations.

Numerical Analysis and Benchmarks

Extensive numerical experiments validate and characterize the framework:

• Validation of Patch Size and Incident Ray Sampling: High-fidelity sensor plane field reconstruction (NCC ≥ 0.99) requires a trade-off between DOE patch size and incident ray count. Larger patches necessitate fewer rays but can incur more curvature-induced angular spectrum errors for conformal DOEs. The authors derive a curvature-error upper bound as $\epsilon_\mathrm{curv} \leq \arcsin(D / 2R)$ and demonstrate empirically that this constraint governs the optimal patch size for a target error budget.
• Monte Carlo Convergence for DOE Scattering: The accuracy of secondary ray sampling from the local angular spectrum is quantified for both simple (metalens) and complex (holographic Siemens star) DOE scattering responses. For multi-lobed (complex) angular spectra, uniform and magnitude-based sampling converge similarly, but for focused spectra (metalens), magnitude-based sampling accelerates convergence.
• Benchmarks on Hybrid Systems: The method reliably reproduces intensity and structural image fidelity for setups where ground truth is known (MSE and NCC values up to $\text{NCC} = 0.9998$), including planar gratings and holographic DOEs, and uniquely extends to hybrid DOE-lens cascades where conventional simulators are inapplicable due to non-paraxial, interference-driven effects.

Inverse Design and End-to-End Optimization

The differentiable nature of the framework enables direct application to inverse design tasks for both planar and conformal hybrid systems:

• End-to-End DOE-Lens Systems: Inverse optimization of a DOE holographic phase profile stacked with a refractive lens, targeting a prescribed sensor intensity, yields a substantial improvement in reconstruction quality ($\text{NCC}$ improvement from 0.421 to 0.934) with suppression of lens-induced non-paraxial artifacts, outperforming paraxial-metalens based design approaches.
• Conformal DOE Optimization: For reflective metasurface DOEs on curved substrates, the approach compensates substrate-induced phase perturbations through gradient-based phase retrieval, achieving quantitative targeting in beam splitting (centroid separation ${\sim}100~\mu\text{m}$, FWHM within diffraction limit) and complex holographic intensity patterns (Stanford "S" image, $\text{NCC}=0.743$) even in the presence of significant surface sag.

Implications and Future Directions

The presented framework enables, for the first time, accurate, general, and differentiable simulation and optimization protocols for optically complex hybrid systems featuring multiple DOEs, high-resolution holography, nontrivial amplitude modulation, and curvilinear geometries. This is particularly significant for emerging computational imaging, immersive display, and freeform nanophotonics, where the ability to rapidly iterate over device architectures and directly incorporate fabrication constraints in optimization loops is crucial.

Theoretical implications include establishing a unified modeling bridge between geometric ray propagation and high-frequency diffractive effects, while practically, the approach can be integrated into existing ray-tracing ecosystems due to its plug-and-play architecture.

Future advancements are anticipated along several axes:

• Adaptive Computational Strategies: Development of adaptive patch sizing and dynamic ray sampling to manage the memory-computation tradeoff, especially for large-scale and multi-layer metasurfaces.
• Integration with Full-Wave Solvers and ML Surrogates: Coupling with FDTD/FEM solvers and physics-augmented neural surrogates to handle scenarios where non-ideal bulk-wave interactions or strong near-field coupling become significant, enabled by the differentiable backbone and advances in neural surrogate modeling [45-47].
• Scaling for Complex Fabrication: As DOEs and hybrid systems increase in complexity (freeform, multi-layer, conformal), such design tools will become critical for robust and manufacturable photonic system design.

Conclusion

This work presents a comprehensive, differentiable ray-wave simulation and inverse design platform for hybrid refractive-diffractive optical systems, overcoming key limitations of conventional methods in handling arbitrary profiles, curvilinear geometries, and non-paraxial effects. By unifying local wave scattering, global ray propagation, and differentiability, the approach enables accurate modeling, robust optimization, and rapid prototyping for a broad class of next-generation computational and display optics (2605.15418). Future extensions in efficiency, physical-model fidelity, and scaling are expected to further consolidate its role in the design of advanced optical hardware.