Ray2Ray: Learned Optical Ray Transfer

• Ray2Ray is a learned proxy ray tracer that uses a multilayer perceptron to directly map input ray origins and directions to output parameters with micron-scale accuracy.
• It replaces traditional iterative surface computations and repeated applications of Snell’s law with a single-pass end-to-end neural network surrogate for complex optical assemblies.
• The method generalizes across unseen configurations and arbitrary target-plane depths, offering consistent positional and angular precision even for large-scale ray sets.

Ray2Ray is a learned proxy ray tracer for optical systems that replaces iterative surface-by-surface ray tracing with a direct neural mapping from input ray origin and direction to output ray origin and direction. Instead of explicitly intersecting the ray with each optical surface and applying Snell’s law repeatedly, it learns the overall transformation induced by the whole optical assembly. In the reported formulation, the model is trained per optical system and acts as a single-pass end-to-end surrogate that preserves ray geometry, predicts output rays in a physically accurate manner from supervised simulator data, and can subsequently be evaluated at arbitrary target-plane depths through simple ray-plane intersection (Sinaei et al., 28 Jul 2025).

1. Concept and problem setting

Ray2Ray addresses a specific inefficiency of conventional optical ray tracing: standard lens-system simulation requires repeated geometric intersection computations and refractive updates at every surface, typically solved via Newton’s method for surface intersections and Snell’s law for refractions. The paper frames this iterative surface-by-surface procedure as computationally expensive when many rays are traced or when the optics are complex (Sinaei et al., 28 Jul 2025).

The defining idea is to learn a system-level ray transfer rather than model individual surfaces explicitly. In that sense, Ray2Ray compresses a fixed optical system into the parameters of a multilayer perceptron and substitutes a single neural-network forward pass for per-surface propagation. This makes Ray2Ray a ray-based implicit neural representation of optical propagation, rather than an explicit geometric solver.

A central conceptual distinction is its separation from point-spread-function surrogates. The paper argues that PSF-based neural models are tied to imaging tasks, target-space sampling schemes, fixed kernel sizes, and often discrete object-space points, whereas Ray2Ray remains ray-based: it preserves both origin and direction of rays, can be queried with arbitrary numbers of rays, at arbitrary resolution, and can support arbitrary target-plane depths without retraining or repeated depth-specific model queries (Sinaei et al., 28 Jul 2025). A plausible implication is that the method is aimed at emulating optical propagation itself rather than only image formation.

2. Ray-space parameterization and learned mapping

In the paper’s formulation, a ray is represented as

$R = (\mathbf{p}, \mathbf{d}),$

where $\mathbf{p}$ is the ray origin and $\mathbf{d}$ is its normalized direction. Ray2Ray models a mapping from an input ray in projection space to an output ray in target space, written as mapping

$$(\mathbf{p}_i, \mathbf{d}_i) \mapsto (\mathbf{p}_o, \mathbf{d}_o).$$

The representation is reduced to 2D position and 2D direction coordinates at input and output: $$\mathbf{p}_i \in \mathbb{R}^2,\quad \mathbf{d}_i \in \mathbb{R}^2,\quad \mathbf{p}_o \in \mathbb{R}^2,\quad \mathbf{d}_o \in \mathbb{R}^2.$$ The network therefore learns a continuous function from a 4D ray parameterization to a 4D output ray parameterization for a given optical system (Sinaei et al., 28 Jul 2025).

The supervised optimization objective is presented, despite typesetting corruption in the source text, as minimizing a discrepancy between the predicted output ray and the ground-truth output ray: $$\hat{w}_{\text{r2r}} = \arg\min_{w_{\text{r2r}}} \mathcal{L}\bigl(\text{Ray2Ray}(p_i,d_i), (p_o^{\text{gt}}, d_o^{\text{gt}})\bigr).$$ The paper does not provide a more explicit decomposition of $\mathcal{L}$ into positional and angular terms, does not specify weighting coefficients or additional regularizers, and does not define any explicit physical constraint term such as refractive consistency, symplectic structure, energy conservation, or aperture masking in the objective (Sinaei et al., 28 Jul 2025).

This means the method is physically grounded through supervised data generated by a conventional optics simulator rather than through explicit physics-based constraints in the neural loss. The paper also does not introduce a more sophisticated canonical coordinate system such as pupil coordinates, normalized field coordinates, Plücker coordinates, or $SE(3)$-invariant ray coordinates, and it does not discuss normalization ranges, encoding schemes, or frequency embeddings. Accordingly, the term “implicit neural representation” is used in the broad sense of an MLP that represents a continuous mapping from ray parameters to ray parameters.

3. Data generation, architecture, and training procedure

For each optical system, Ray2Ray is trained on paired input-output rays synthesized by a standard optics simulator. The source is modeled as an $M \times M$ grid, and the input ray origin $\mathbf{p}_i$ is the center of a grid cell in projection space. For each grid cell, the method generates 1024 rays. Input directions $\mathbf{p}$0 are sampled using Monte Carlo sampling at the entrance of the optical setup. All rays use a fixed wavelength of $\mathbf{p}$1, so the trained model is monochromatic and does not model wavelength-dependent effects such as chromatic aberration unless separate training is performed (Sinaei et al., 28 Jul 2025).

Ground truth is generated with the odak optics toolkit. Rays are traced through the optical system, and their intersections with a target plane yield output positions $\mathbf{p}$2 and directions $\mathbf{p}$3. The training target plane is placed very close to the output side of the optical system, at $\mathbf{p}$4 in target space, which the authors state improves convergence during training. After training, the predicted ray can be intersected with arbitrary target planes, which is the basis for arbitrary-depth evaluation (Sinaei et al., 28 Jul 2025).

The neural architecture is a simple multilayer perceptron with residual skip connections inserted every three layers to stabilize training. Each hidden layer uses a ReLU activation. The paper reports an ablation study across nine off-the-shelf optical systems with different MLP depths and widths to choose an appropriate architecture, but the actual architectural numbers are not included in the provided main text and are said to be in supplementary material (Sinaei et al., 28 Jul 2025).

Training uses AdamW for 3000 epochs. The initial learning rate is $\mathbf{p}$5, and it decays exponentially during training. The train/test split is performed at the source-grid-cell level: $\mathbf{p}$6 of grid cells are used for training and $\mathbf{p}$7 for testing, so the test set contains unseen source positions (Sinaei et al., 28 Jul 2025).

4. Inference behavior and reported empirical performance

At inference time, a new ray defined in projection space by its source location and direction is mapped directly to output ray parameters $\mathbf{p}$8 near the optical exit. To evaluate the optical system on any downstream target plane, the predicted output ray is intersected with that plane. This operationally separates learning the near-exit transfer map from downstream geometric evaluation (Sinaei et al., 28 Jul 2025).

The paper evaluates Ray2Ray on nine off-the-shelf optical systems overall. The detailed setup described in the provided text is a customized single-lens optical system with aperture size $\mathbf{p}$9, focal length $\mathbf{d}$0, and a source grid spanning $\mathbf{d}$1, divided into $\mathbf{d}$2 cells. Across the trained systems, the reported accuracy is positional errors on the order of $\mathbf{d}$3 and angular deviations on the order of $\mathbf{d}$4 in estimated output rays (Sinaei et al., 28 Jul 2025).

Two explicit generalization experiments are reported for the single-lens system. On a previously unseen grid cell, the model yields positional error $\mathbf{d}$5 and angular error $\mathbf{d}$6. On a novel ray pattern containing $\mathbf{d}$7 rays that differs from the training distribution, it yields positional error $\mathbf{d}$8 and angular error $\mathbf{d}$9 (Sinaei et al., 28 Jul 2025).

These results support the paper’s claim that the ray-based representation is not tied to a particular image resolution, ray count, or kernel support. This suggests that Ray2Ray generalizes both across unseen source positions and to much denser ray sets than those used during training, while retaining a near-exit representation that can be propagated to arbitrary downstream planes without depth-specific retraining.

5. Scope, assumptions, and methodological limits

Ray2Ray is trained per optical system. The network weights encode the behavior of one fixed optical system, so changing the lens design, spacing, material, or alignment requires generating new training data and retraining. The present form is also trained at a single wavelength, $$(\mathbf{p}_i, \mathbf{d}_i) \mapsto (\mathbf{p}_o, \mathbf{d}_o).$$0, making it monochromatic (Sinaei et al., 28 Jul 2025).

The supervision target is defined at a plane $$(\mathbf{p}_i, \mathbf{d}_i) \mapsto (\mathbf{p}_o, \mathbf{d}_o).$$1 from the optical system, so the method relies on this near-exit-plane parameterization being easy to learn. Although arbitrary downstream depths are supported afterward by geometric intersection, the paper does not deeply analyze whether errors grow with target distance (Sinaei et al., 28 Jul 2025).

The formulation also appears limited to systems and rays that can be well represented by a single-valued mapping from input ray parameters to one output ray. The paper identifies this as appropriate for conventional refractive systems but potentially problematic in cases with scattering, severe vignetting ambiguities, diffraction-dominated behavior, or multi-path optical phenomena. None of these are modeled explicitly. Since the loss function is not physically constrained, physical validity beyond the training distribution depends on training-data coverage (Sinaei et al., 28 Jul 2025).

The evidence presented for efficiency is also limited in scope. The method is framed as “efficient” and “single pass,” but the provided text does not include explicit runtime benchmarks, speedup factors, throughput, memory usage, or latency comparisons against a conventional optical ray tracer. Likewise, there are no direct quantitative baselines against PSF-based neural models, alternative learned ray mappings, spline models, polynomial ray-transfer approximations, or analytical paraxial methods (Sinaei et al., 28 Jul 2025). The reported evidence is therefore accuracy-focused rather than a full time-performance study.

6. Position within the broader ray-mapping literature

Within the immediate neural-optics context, Ray2Ray is distinguished by learning a direct ray-to-ray transfer map rather than a point-spread function. More broadly, several adjacent research lines also operate on mappings between ray spaces or on ray-state transformations, but they address different regimes and make different modeling commitments.

One neighboring direction is phase-space ray-bundle transport. “Concatenated Backward Ray Mapping on the Compound Parabolic Concentrator” (Jansen et al., 2023) defines deterministic maps between source and target phase-space regions for 2D reflective systems, using propagation and reflection maps concatenated path by path. Another neighboring direction is wave-optical ray-state transport. “A Generalized Ray Formulation For Wave-Optics Rendering” (Steinberg et al., 2023) defines generalized rays as detector-derived Gaussian states in phase space and transports them backward through local interaction operators. “Ray Theory of Waves” (Ren et al., 2024) treats a ray as carrying wave vector, complex electric vector, and wavefront curvature, and derives update laws across interfaces. These approaches are conceptually adjacent in that they move beyond purely local point-ray tracing, but they are not the same method as Ray2Ray.

A distinct possible source of confusion is nomenclature. Ray2Ray in optics should be distinguished from the “ray-by-ray+” transport simplification in core-collapse supernova simulations, which is a dimensional-reduction approximation for neutrino transport rather than a learned optical surrogate (Skinner et al., 2015). It should also be distinguished from the C$$(\mathbf{p}_i, \mathbf{d}_i) \mapsto (\mathbf{p}_o, \mathbf{d}_o).$$2-Ray and pyC$$(\mathbf{p}_i, \mathbf{d}_i) \mapsto (\mathbf{p}_o, \mathbf{d}_o).$$3Ray radiative-transfer code family, which concerns photon-conserving short-characteristics transport for cosmic reionization rather than neural ray transfer in lens systems [(Friedrich et al., 2012); (Hirling et al., 2023)].

In that broader context, Ray2Ray is most precisely understood as a learned, per-system, continuous ray-transfer surrogate for optical systems. Its central contribution is to show that a compact MLP-based implicit representation can approximate optical ray propagation with micron-scale positional accuracy and hundredths-of-a-degree angular accuracy, while preserving the operational advantages of a ray-based representation: arbitrary ray counts, arbitrary resolution, and arbitrary downstream target-plane depths after a single near-exit prediction (Sinaei et al., 28 Jul 2025).