Differentiable Ray-Wave Framework

• Differentiable Ray-Wave Framework is a model architecture that couples ray-based transport with wave propagation, allowing gradients to flow to physical parameters for optimized imaging and simulation.
• It integrates techniques like phase recovery via wavefront curvature, Fourier-domain transforms, and Monte Carlo methods to bridge ray and wave formulations in various domains.
• Key applications include improving optical design, calibrating radio and ultrasound simulations, while addressing challenges related to discrete path topology and regime transitions.

A differentiable ray-wave framework is a modeling architecture in which ray-based transport and wave-based field propagation are composed inside an automatic-differentiation graph, so that gradients of images, channels, or task losses can be propagated to physical parameters. In the current literature, this idea appears as a hybrid ray–wave forward model for compound optics, a differentiable ray tracer for radio propagation that carries complex amplitude, phase, and polarization, and a full-path Monte Carlo ultrasound simulator that couples ray transport to beamforming and image formation (Ho et al., 2024, Hoydis et al., 2023, Spencer et al., 16 Apr 2026).

1. Foundational ideas

A foundational mathematical view treats waves, rays, and diffusion as governed by the same spatial co-metric $g^{ij}(\mathbf r)$: the wave equation uses $g^{ij}$ in the second-order spatial operator, while the ray limit yields geodesics of the inverse metric $g_{ij}$ (Kinsler et al., 2015). In a more explicitly wave-aware ray model, the Ray Theory of Waves augments each ray with a wavevector, a complex electric field vector, and a curvature matrix $\mathbf Q$, so that amplitude and phase are recovered from wavefront curvature rather than from purely geometric spot tracing (Ren et al., 2024). Adjacent differentiable ray formulations supplied another precursor abstraction: differentiable ray consistency for single-view 3D reconstruction defined per-ray losses as expectations over probabilistic termination events and showed that the resulting sums and products are fully differentiable with respect to voxel occupancies and attributes (Tulsiani et al., 2017).

Taken together, these formulations establish the conceptual core of the field. Rays provide geometric structure, path ordering, and transport skeletons; wave descriptions provide phase, diffraction, interference, or curvature; and differentiability turns the combined model into an optimization operator. This suggests that a differentiable ray-wave framework is less a single algorithm than a family of compatible constructions that share three properties: a ray-space parameterization, a wave- or field-level augmentation, and an end-to-end gradient path.

2. Ray–wave coupling operators

In computational imaging with compound optics, the forward operator is written as

$$I(\mathbf u) = \int b(\mathbf x)\, h(\mathbf u \mid \mathbf x)\, d\mathbf x,$$

with a differentiable ray tracer through multiple refractive surfaces used to compute optical path lengths

$\delta_i = \int_{C_i} n(s)\, ds$

and an exit-pupil field

$v(\boldsymbol\rho_i) = a_i \exp(jk\delta_i).$

That field is then propagated to the sensor by a Monte Carlo Rayleigh–Sommerfeld integral to form point spread functions, yielding an explicit ray–wave bridge from lens prescription to sensor irradiance (Ho et al., 2024).

In hybrid refractive-diffractive imaging, coherent ray tracing to a DOE plane is followed by pointwise DOE phase modulation and angular-spectrum propagation to the sensor. The field after the DOE is

$$\mathbf U_{\text{DOE}^{+}} = \mathbf U_{\text{DOE}^{-}} \exp\!\left(j\frac{2\pi}{\lambda}(n_\lambda-1)h(x,y)\right),$$

and the PSF is the sensor-plane intensity $|\mathbf U_{\text{Sensor}}|^2$ (Yang et al., 2024). A related hybrid optical pipeline for metalenses uses two explicit wave-to-ray bridges: a windowed Fourier transform, which converts a local field into a directional spectrum, and a phase-gradient method, which sets the local wavevector by $$\mathbf k(\mathbf r)=\nabla S_{\text{out}}(\mathbf r)$$ and launches one ray per sample (Zhu et al., 2023). A more general hybrid optical formulation treats each ray–DOE interaction locally: a finite patch of a planar or curvilinear DOE is transformed by the angular spectrum method, propagating Fourier components are sampled, and secondary rays are assigned amplitudes

$g^{ij}$0

thereby embedding arbitrary holographic diffractive profiles into standard ray-tracing pipelines (Cheng et al., 14 May 2026).

Outside optics, the same coupling pattern recurs. In radio propagation, the channel is represented as a multipath ray sum,

$g^{ij}$1

where ray geometry determines delays and EM interactions determine complex amplitudes, phase shifts, and polarization transforms (Hoydis et al., 2023). In ultrasound, directional pressure is propagated with a rendering-equation-like transport law, each valid path contributes an impulse at a time of flight, and those pathwise contributions are converted into RF data, delay-and-sum beamforming, envelope detection, and log compression (Spencer et al., 16 Apr 2026). Across these cases, the common mechanism is explicit: rays define where and how fields are evaluated, while wave operators determine what those fields do.

3. Differentiability and optimization

In end-to-end computational imaging, the physical model is written as $g^{ij}$2, the downstream algorithm as $g^{ij}$3, and gradients $g^{ij}$4 and $g^{ij}$5 are obtained automatically through ray tracing, OPL summation, complex exponentials, diffraction, PSF interpolation, and convolution (Ho et al., 2024). The same design appears in radio differentiable ray tracing, where path discovery is discrete and externally performed, but once the path set is fixed, channel impulse responses, path gains, phases, delays, coverage maps, and derived metrics are differentiable with respect to material parameters, antenna patterns, array geometry, and transmitter or receiver pose (Hoydis et al., 2023).

More specialized differentiable ray solvers apply implicit differentiation to the optimization or algebraic subproblems that define rays. A GPU-accelerated formulation for mixed reflection–diffraction Fermat paths rewrites path finding as convex minimization of total path length and differentiates the optimum implicitly, which the reported experiments show to be about ten times faster than differentiating through solver iterations directly (Eertmans et al., 17 Oct 2025). In image-centered ray shooting for gravitational microlensing, polynomial roots are differentiated implicitly and custom JVPs are supplied for Heaviside-like source-membership tests, yielding a fully differentiable ray solver suitable for Hamiltonian Monte Carlo and variational inference (Miyazaki et al., 3 Oct 2025). In mmWave propagation over noisy reconstructed geometry, mmDiff replaces noise-sensitive specular reflection by a differentiable directional scattering kernel,

$g^{ij}$6

and proves that the approximation preserves asymptotic path-gain accuracy in the specular limit (Lu et al., 26 May 2026).

The technical significance is that differentiability is rarely obtained by differentiating every discrete decision. Instead, frameworks either fix path topology during a forward–backward pass, regularize singular interactions into smooth kernels, or differentiate the optimality conditions of lower-level solvers. This is a recurring structural choice across optics, radio, and astrophysical ray shooting.

4. Approximations, sampling, and scalability

Physical fidelity and computational cost are balanced through explicit approximations. In compound-lens imaging, a full shift-variant wave-optical renderer is made tractable by a locally isoplanatic approximation that reparametrizes the scene in sensor coordinates and replaces the imaging operator with a sum of convolutions,

$g^{ij}$7

For a Cooke triplet, using 9 PSFs instead of 969 gives approximately $g^{ij}$8 speedup in forward propagation and approximately $g^{ij}$9 in backward propagation, with minimal loss in performance (Ho et al., 2024).

Patchwise wave solvers introduce another layer of approximation. In the general hybrid optical framework, each ray interacts only with a finite DOE patch, and for curved surfaces the tangent-plane approximation incurs a bounded angular-spectrum error

$g_{ij}$0

where $g_{ij}$1 is patch size and $g_{ij}$2 is local radius of curvature (Cheng et al., 14 May 2026). The number of secondary sampled rays per patch then controls Monte Carlo convergence; the paper reports that sampling proportional to $g_{ij}$3 converges substantially faster than uniform sampling for concentrated spectra, while both behave similarly for complex multi-lobed holograms.

Wave-based and path-based simulators adopt parallel strategies for memory control. Fully differentiable ultrasound uses micro-batching over steering angles and subsets of rays or elements, and fixes random seeds across optimization steps as common random numbers to reduce gradient variance (Spencer et al., 16 Apr 2026). SWEEP, a differentiable seismic wave-equation framework, organizes propagators, sources, and receivers in a plug-and-play architecture and uses batch modeling and multi-GPU execution through PyTorch or JAX so that multiple shots or multiple models can be simulated in parallel (Wang et al., 1 Apr 2026). These choices indicate that scalability in differentiable ray-wave systems usually depends less on a single solver and more on how one decomposes the forward model into differentiable, vectorizable blocks.

5. Representative applications

In optical computational imaging, wave-aware training changes both physical designs and learned algorithms. A differentiable wave-optics model for compound optics showed that systems optimized only with ray optics degrade when wave optics is introduced at test time, and reported a classification example in which aperture $g_{ij}$4 mm and $g_{ij}$5m pixels yielded $g_{ij}$6 for wave-trained systems versus $g_{ij}$7 for ray-trained systems under wave-optics evaluation (Ho et al., 2024). End-to-end hybrid refractive-diffractive lens design further demonstrated accurate PSF prediction beyond paraxial or local-grating approximations, joint optimization of refractive lens parameters, DOE parameters, and a reconstruction network, and real-world improvements in aberration correction and extended depth-of-field imaging (Yang et al., 2024). Hybrid metalens–lens optimization and patchwise DOE scattering models extend the same principle to nonparaxial imaging, chromatic aberration correction, and planar or conformal holographic surfaces (Zhu et al., 2023, Cheng et al., 14 May 2026).

In radio propagation, differentiable ray tracing has been used both as a forward simulator and as an inverse-modeling tool. Sionna RT learns radio-material parameters from channel responses and optimizes transmitter orientation by differentiating through path gains, phases, delays, and coverage maps (Hoydis et al., 2023). DiffSBR uses a differentiable shooting-and-bouncing-rays engine to refine scene parameters against measured radar point clouds for mmWave-based 3D reconstruction (Chen et al., 2023). mmDiff calibrates spatially varying material models from sparse AoA power spectra on noisy reconstructed meshes by replacing pure specular reflection with a directional scattering approximation that is robust to local geometric artifacts (Lu et al., 26 May 2026).

In ultrasound, UltraRay introduced a full-path ray tracing pipeline that traces each path from the transducer through the scene and back to the sensor, then applies a standard signal-processing chain to generate B-mode images (Duelmer et al., 10 Jan 2025). A later fully differentiable ultrasound simulator propagated gradients from image-space losses back through acoustic transport, beamforming, and post-processing, recovering known parameters in a simulated-reference setting and matching experimental B-mode data through effective parameter estimation in a simulation-to-real setting (Spencer et al., 16 Apr 2026). Adjacent ray-centric inverse problems, such as single-view 3D reconstruction with differentiable ray consistency and fully differentiable microlensing magnification, show that the same architectural pattern extends well beyond conventional imaging hardware.

6. Limitations, discontinuities, and open problems

The main limitations are remarkably consistent across domains. Optical frameworks based on scalar diffraction ignore polarization and vectorial effects, and several models operate with a single wavelength per color channel or assume locally isoplanatic blur fields (Ho et al., 2024). Hybrid refractive-diffractive imaging models that place the DOE near the sensor remain expensive because coherent ray tracing, high-resolution wave propagation, and backpropagation all require large memory, and one framework explicitly notes that it does not support wave-to-ray conversion after a mid-train DOE (Yang et al., 2024). Metalens-enhanced ray optics similarly neglects volumetric diffraction between surfaces and relies mainly on the phase-gradient method when a full local directional spectrum would be too costly (Zhu et al., 2023).

Radio and ultrasound frameworks face a different but structurally related obstacle: path topology changes. Sionna RT treats path discovery as non-differentiable discrete logic and computes gradients only conditionally on a fixed path set (Hoydis et al., 2023). The same issue appears in reflection–diffraction path optimization, in differentiable radar ray tracing, and in ultrasound path tracing, where visibility changes, shadow boundaries, or new multi-bounce paths create piecewise-smooth rather than globally smooth forward maps (Eertmans et al., 17 Oct 2025, Chen et al., 2023, Spencer et al., 16 Apr 2026). The comparison between differentiable and dynamic ray tracing makes this explicit: dynamic extrapolation is valid only inside multipath cells where the set of valid path candidates is constant, and the proposed Multipath Lifetime Map quantifies those regions through cell areas and average minimal inter-cell distances (Eertmans et al., 2024).

A plausible implication is that the central open problem is no longer differentiability in the narrow sense, but differentiability across regime changes: across visibility events, across caustics, across transitions between smooth phase profiles and strongly diffractive holograms, and across high-frequency approximations and full-wave physics. Current frameworks already provide the computational grammar—ray transport, field augmentation, differentiable propagation, and end-to-end losses. The next step is to make those ingredients robust when the underlying path structure itself changes.