Geometric Ray Modeling in 3D Rendering & Optics

• Geometric Ray Modeling (GRM) is a methodological principle that treats rays as primary geometric primitives to explicitly encode intersections, occlusion, and transport for accurate scene reconstruction.
• GRM is applied across neural rendering, layered reconstruction, LiDAR simulation, and optical modeling, using parameterized ray equations and geodesic formulations to resolve volumetric ambiguities.
• By imposing ray-wise structure, GRM improves pose estimation and view synthesis while ensuring robust handling of occlusion, layered intersections, and complex light transport.

Searching arXiv for papers on "Geometric Ray Modeling" and related formulations across rendering, 3D reasoning, diffusion-based reconstruction, and gesture grounding. Geometric Ray Modeling (GRM) denotes a family of formulations in which rays are treated as the primary geometric primitive rather than as a by-product of pointwise or volumetric representations. In the cited literature, GRM is used to model scene geometry and appearance explicitly along rays for novel view synthesis, to treat 3D geometry as the set of intersections between rays and surfaces, to represent camera poses as neural bundle rays, to encode hand pointing orientation as an explicit ray, and to describe propagation as ray or geodesic evolution in optical and Riemannian settings (Yang et al., 2023, Li et al., 25 Apr 2025, Chen et al., 28 Mar 2025, Li et al., 23 Jun 2026, Chávez-Islas et al., 21 Jun 2026, Coulon et al., 2020). Taken together, these usages suggest that GRM is less a single algorithm than a recurring methodological principle: sequential structure, occlusion ordering, visibility, intersections, and transport are imposed directly in ray space.

1. Foundational formulations and geometric primitives

A central GRM construction is the parameterized ray. In neural rendering, a camera ray is written as $r(t)=o+t d$, and standard NeRF rendering computes pixel color by integrating density $\sigma$ and color $c$ along that ray, with

$$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$

In single-view layered reconstruction, LaRI adopts the pinhole camera model with $d_c\propto K^{-1}\tilde p$, $d_w=R d_c$, $o_w=-R^\top t$, and world-space ray $r_w(\tau)=o_w+\tau d_w$, while in camera coordinates the ray is $r_c(\tau)=\tau d_c$ (Yang et al., 2023, Li et al., 25 Apr 2025).

Optics-oriented GRM uses the same primitive in a different parameterization. In freeform gradient-index media, rays are space curves $\mathbf r(s)$ parameterized by arc length $\sigma$0, governed by the eikonal relations

$\sigma$1

In the Riemannian formulation of ray-marching in the eight Thurston geometries, rays are geodesics $\sigma$2 satisfying

$\sigma$3

and in the geometrical theory of diffracted rays they are geodesics of an optical metric induced by refractive index or potential (Chávez-Islas et al., 21 Jun 2026, Coulon et al., 2020, Micheli et al., 2013).

These formulations share a common structural move. GRM makes the ordered samples or intersections along a ray explicit, so that ambiguities that are diffuse in volumetric or token-based models can be constrained as ray-wise selection, layered validity, geometric consistency, or transport along a geodesic. This interpretation is explicit in CeRF, LaRI, GCRayDiffusion, and VistaRef, even though the domains differ substantially (Yang et al., 2023, Li et al., 25 Apr 2025, Chen et al., 28 Mar 2025, Li et al., 23 Jun 2026).

2. Ray-structured neural rendering and geometric disambiguation

In novel view synthesis, GRM is used to address the underconstrained nature of volumetric rendering. CeRF identifies a “geometric ambiguity” in standard NeRF: many different $\sigma$4 profiles can produce similar accumulated transmittance $\sigma$5 and hence nearly identical colors, while light-field models can explain a ray’s color equally well by a point on the true surface $\sigma$6 and a nearby off-surface point $\sigma$7 when a limited set of viewpoints produces indistinguishable observations (Yang et al., 2023).

CeRF operationalizes GRM by modeling the derivative of outgoing radiance along a ray, $\sigma$8, rather than $\sigma$9 directly. The reconstruction relation is

$c$0

The physical intuition stated in the paper is that along a ray passing through empty space, radiance is constant until the first surface interaction, so the derivative along $c$1 is sparse and concentrated near surface intersections. CeRF approximates this sparsity with

$c$2

and in the discrete setting writes

$c$3

where $c$4 approximates a Dirac delta at the true intersection sample(s) (Yang et al., 2023).

The architecture is explicitly ray-structured. The Convolutional Ray Feature Extractor $c$5 is a U-shaped 1D CNN over the ordered sample sequence $c$6. Early $c$7 convolutions act like per-sample MLPs, downsampling convolutions with kernel size $c$8 and stride $c$9 capture local neighborhoods and enlarge the receptive field, and upsampling with skip connections restores per-sample features. A Geometry Attribute Network $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$0 with a GRU then propagates state along the ray in sample order, imposing a causal, occlusion-aware prior in which earlier samples influence later decisions. A small MLP maps the GRU hidden state to raw geometry coefficients $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$1, and the Unique Surface Constraint

$$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$2

normalizes selection weights along the ray, including an epipolar background point at infinity with raw coefficient $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$3 and background color $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$4. Final rendering uses the epipolar expectation

$$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$5

This replaces NeRF’s $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$6 integral with a learned, ray-structured selection-and-sum (Yang et al., 2023).

Training combines photometric loss over coarse and fine rays with Empty Space Regularization,

$$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$7

with $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$8 for the coarse stage and $$C(r)=\int_{t_n}^{t_f} T(t)\sigma(t)c(t)\,dt,\qquad T(t)=\exp\!\left(-\int_{t_n}^{t}\sigma(s)\,ds\right).$$9 for the fine stage, Adam with learning-rate annealing from $d_c\propto K^{-1}\tilde p$0 to $d_c\propto K^{-1}\tilde p$1, batch size $d_c\propto K^{-1}\tilde p$2, and $d_c\propto K^{-1}\tilde p$3k iterations on $d_c\propto K^{-1}\tilde p$4 RTX 3080 Ti. On Blender, CeRF reports $d_c\propto K^{-1}\tilde p$5 in PSNR/SSIM/LPIPS, compared with Ref-NeRF at $d_c\propto K^{-1}\tilde p$6 and Mip-NeRF at $d_c\propto K^{-1}\tilde p$7. On Shiny Blender, CeRF reports $d_c\propto K^{-1}\tilde p$8, ranks second overall, and outperforms NeRF baselines especially on subsurface and complex reflections such as coffee and teapot scenes (Yang et al., 2023).

The ablations are important because they isolate the GRM components. On the drums scene, removing the Convolutional Ray Feature Extractor reduces PSNR from $d_c\propto K^{-1}\tilde p$9 to $d_w=R d_c$0, removing the GRU-based Geometry Attribute Network gives $d_w=R d_c$1, removing the Unique Surface Constraint yields the worst result at PSNR $d_w=R d_c$2 and LPIPS $d_w=R d_c$3, removing the epipolar expectation gives $d_w=R d_c$4, and removing Empty Space Regularization slightly changes metrics but hurts geometry at infinity as validated via depth visualization. The reported limitations are equally specific: specular or mirror-like BRDFs are not explicitly modeled, compute and memory footprint are higher due to conv and RNN, and performance depends on sampling and hyperparameters such as $d_w=R d_c$5 and $d_w=R d_c$6 (Yang et al., 2023).

3. Layered intersections, view-aligned point maps, and ray-based pose inference

LaRI instantiates GRM for single-view inference by predicting, for every image pixel, all surface intersection points along the corresponding camera ray, ordered by depth, in a compact, view-aligned form (Li et al., 25 Apr 2025). Its Layered Point Maps are

$d_w=R d_c$7

where $d_w=R d_c$8 is the $d_w=R d_c$9-th ray-surface intersection in camera coordinates. Rather than enforcing ordering by an explicit ordinal loss, LaRI predicts a ray stopping index using logits $o_w=-R^\top t$0 and

$o_w=-R^\top t$1

with $o_w=-R^\top t$2 meaning “no intersection” and $o_w=-R^\top t$3 meaning layers $o_w=-R^\top t$4 are valid. The derived mask

$o_w=-R^\top t$5

enforces contiguous validity from first hit to stopping layer (Li et al., 25 Apr 2025).

The network uses a two-head design: a ViT-Large backbone followed by a CNN decoder for dense regression of $o_w=-R^\top t$6, and a separate ViT-Large plus dense segmentation decoder for the stopping logits $o_w=-R^\top t$7. Training uses scale–shift alignment,

$o_w=-R^\top t$8

then a point-map loss $o_w=-R^\top t$9 and a ray-stopping cross-entropy loss $r_w(\tau)=o_w+\tau d_w$0. The data-generation pipeline combines Objaverse v1, 3D-FRONT rooms, and ScanNet++, with Blender for photorealistic RGB rendering and PyTorch3D for ray-traced layered intersections and masks. Efficiency is a central result: the object-level model uses about $r_w(\tau)=o_w+\tau d_w$1M parameters and about $r_w(\tau)=o_w+\tau d_w$2 ms per inference, versus TRELLIS at about $r_w(\tau)=o_w+\tau d_w$3M parameters and about $r_w(\tau)=o_w+\tau d_w$4 ms (Li et al., 25 Apr 2025).

On object-level, view-aligned GT evaluation, LaRI reports $r_w(\tau)=o_w+\tau d_w$5, $r_w(\tau)=o_w+\tau d_w$6, $r_w(\tau)=o_w+\tau d_w$7, and $r_w(\tau)=o_w+\tau d_w$8, surpassing TRELLIS at $r_w(\tau)=o_w+\tau d_w$9, $r_c(\tau)=\tau d_c$0, $r_c(\tau)=\tau d_c$1, and $r_c(\tau)=\tau d_c$2. On scene-level SCRREAM, LaRI reports visible-surface $r_c(\tau)=\tau d_c$3, unseen-surface $r_c(\tau)=\tau d_c$4, and overall $r_c(\tau)=\tau d_c$5, with occluded geometry recovered in one feed-forward. The cited limitations are fewer points on surfaces nearly parallel to rays and in inter-layer gaps, sensitivity to textureless regions, specular and transparent materials, and thin structures, plus deterministic underfitting for severely occluded objects when visual cues are insufficient (Li et al., 25 Apr 2025).

GCRayDiffusion applies GRM to pose-free surface reconstruction from unposed, sparse-view images by representing each camera as a set of neural bundle rays

$r_c(\tau)=\tau d_c$6

where $r_c(\tau)=\tau d_c$7 is a unit direction, $r_c(\tau)=\tau d_c$8 is a moment term, and $r_c(\tau)=\tau d_c$9 is an endpoint depth. The explicit on-surface sample is

$\mathbf r(s)$0

Forward diffusion on ray parameters uses

$\mathbf r(s)$1

and the denoiser is conditioned on image features and a global triplane-based signed distance field $\mathbf r(s)$2, with loss

$\mathbf r(s)$3

The surface regularizer is

$\mathbf r(s)$4

and the total objective is $\mathbf r(s)$5 (Chen et al., 28 Mar 2025).

This construction turns camera pose estimation into a geometry-aware diffusion problem. The reported empirical results on Objaverse include rotation accuracy $\mathbf r(s)$6 for image counts $\mathbf r(s)$7, translation accuracy $\mathbf r(s)$8, and surface metrics $\mathbf r(s)$9, $\sigma$00, $\sigma$01, and $\sigma$02. On GSO, the method reports $\sigma$03, $\sigma$04, $\sigma$05, and $\sigma$06 (Chen et al., 28 Mar 2025).

4. GRM as explicit orientation encoding in visual grounding

VistaRef uses GRM in a markedly different setting: pointing-to-object detection in natural images. Here GRM is the module that turns implicit hand pointing cues into explicit, differentiable geometric features that guide attention and feature aggregation toward the correct referent (Li et al., 23 Jun 2026). The backbone is a frozen BEiT-3 model producing visual patch features $\sigma$07 and a linguistic sequence $\sigma$08, from which text-guided visual aggregation forms

$\sigma$09

A keypoint head then predicts normalized 2D hand root and fingertip coordinates,

$\sigma$10

From these, GRM computes the displacement $\sigma$11, the length $\sigma$12, and the unit direction $\sigma$13 (Li et al., 23 Jun 2026).

The geometric descriptor is a $\sigma$14D vector

$\sigma$15

which is projected by a three-layer Ray Encoder to $\sigma$16. This feature does not use camera intrinsics, extrinsics, depth cues, Plücker coordinates, or a parametric $\sigma$17; the geometry is purely image-plane oriented. The ray embedding is fused with Local Hand Entity Modeling features via

$\sigma$18

used in cross-attention,

$\sigma$19

and injected into the final regression features

$\sigma$20

with $\sigma$21 and $\sigma$22 (Li et al., 23 Jun 2026).

Training combines the base detection loss

$\sigma$23

with the Orientation-Consistent Alignment Loss

$\sigma$24

under asymmetric supervision that disables $\sigma$25 and $\sigma$26 on negative samples. The abstract reports a $\sigma$27-point absolute gain in grounding accuracy. The module ablations show that the BEiT-3-Base baseline reports $\sigma$28 and $\sigma$29, while GRM+LHEM+Cross-Attn reaches $\sigma$30 and $\sigma$31. The reported failure modes are very long pointing distances or low hand resolution, where small pose errors are magnified into ray deviation, and cases with multiple overlapping objects, where the ray geometry can be correct but semantic discrimination remains insufficient (Li et al., 23 Jun 2026).

5. Geodesic, optical, and wave-informed ray models

In geometrical optics, GRM is the description of light propagation by ray trajectories and their interactions with media and boundaries. A recent vectorial formulation for arbitrary freeform gradient-index media models ray trajectories as a succession of local refractions on isoindicial surfaces using the vectorial form of Snell’s law together with constant optical path length stepping,

$\sigma$32

For refraction, with $\sigma$33 and $\sigma$34, the transmitted direction is

$\sigma$35

while total internal reflection uses

$\sigma$36

The method handles volume bending, refraction at external boundaries, and total internal reflection in a single vectorial framework. On the analytical benchmark of a $\sigma$37D parabolic-index cylindrical GRIN fiber with $\sigma$38, $\sigma$39, $\sigma$40, $\sigma$41, and $\sigma$42 steps, the reported transverse RMSE is $\sigma$43 and runtime is $\sigma$44 s in MATLAB R2024b on Apple M4 with $\sigma$45 GB RAM (Chávez-Islas et al., 21 Jun 2026).

The same geometric-optics viewpoint appears in step-index multimode fibers, where skew rays do not intersect the fiber axis but reflect at successive azimuths and advance axially by a constant amount. Using Geometric Algebra, the propagation law is

$\sigma$46

the reflection law is

$\sigma$47

and the result is a polygonal helical path characterized by three invariants: constant ray path distance, constant difference in axial distances, and constant difference in the azimuthal angles. The paper also rederives the generalized numerical aperture for skew rays, which reduces to the standard form for meridional rays (Ang et al., 2015).

A more general geodesic GRM appears in real-time rendering of the eight Thurston geometries and of quotient manifolds and orbifolds. There, each screen pixel defines a unit tangent vector at the camera point, which determines an arc-length parameterized geodesic $\sigma$48 through the scene. Ray-marching proceeds by stepping forward by the signed distance to the scene, and quotient spaces are handled by “teleporting” points and rays across face pairings of a fundamental domain. Lighting is adapted to non-Euclidean settings by making light intensity inversely proportional to the area density $\sigma$49 of the geodesic sphere image at distance $\sigma$50 in direction $\sigma$51 (Coulon et al., 2020).

The geometrical theory of diffracted rays extends GRM beyond classical geometrical optics by treating rays as geodesics in a Riemannian manifold with boundary. The eikonal equation

$\sigma$52

is coupled to a transport equation, and near fold caustics the Chester–Friedman–Ursell transformation yields the Airy uniform approximation

$\sigma$53

The associated Ludwig system is mixed-type—hyperbolic for $\sigma$54, parabolic at $\sigma$55, and elliptic for $\sigma$56—and complex rays in the shadow of the caustic describe evanescent waves, creeping waves, and orbiting resonances (Micheli et al., 2013).

Wave-informed GRM is also central to the $\sigma$57D Vectorial Complex Ray Model for generalized rainbow patterns of oblate drops. VCRM3D carries vectorial polarization and intrinsic wavefront curvature along each ray, so that amplitudes and phases of emergent rays are predicted through Fresnel reflection and transmission, coordinate-frame rotations, the divergence factor determined by the Gaussian curvature of the wavefront, and the focal-line phase. With about $\sigma$58 million incident rays, Debye orders $\sigma$59, and an observation grid of $\sigma$60 points over $\sigma$61 and $\sigma$62, the reported runtime is about $\sigma$63 minutes on a desktop. The simulations reproduce the skeleton, coarse, and fine structures of generalized rainbow patterns, including hyperbolic-umbilic caustics for aspect ratios such as $\sigma$64 and $\sigma$65, and an HU focus at $\sigma$66 (Duan et al., 2021).

6. Emitter-centric filtering, tomographic projection, and current directions

GRM is also used for efficient sensing and inverse problems in which rays are known a priori and geometry must be filtered or integrated relative to them. In arbitrarily dynamic LiDAR simulation, the Gajmer Ray-Casting Algorithm replaces a large set of discrete rays by emitter-centric geometric loci. For a spinning multi-channel LiDAR with emitter position $\sigma$67, spin axis $\sigma$68, channel elevations $\sigma$69, and azimuth samples $\sigma$70, each ray is

$\sigma$71

Over a full azimuth sweep, nonzero-elevation channels trace cones around $\sigma$72, while zero-elevation channels degenerate to planes. GRCA asks which rays can each triangle possibly hit, not what each ray hits, and uses cone or plane swept surfaces plus apparent-area and solid-angle bounds to filter channel eligibility and azimuth intervals before any ray-triangle test. Its overall complexity is

$\sigma$73

in contrast to BVH-style $\sigma$74 plus rebuild or refit cost. In benchmarks with $\sigma$75–$\sigma$76 simultaneous $\sigma$77-ray LiDARs, GRCA reaches up to $\sigma$78 over hardware-accelerated OptiX and $\sigma$79 over Embree without range culling; with realistic deployment ranges of $\sigma$80–$\sigma$81 m it reaches up to $\sigma$82 GPU and $\sigma$83 CPU; and in a hybrid pipeline with static BVH handling it reaches up to $\sigma$84 GPU and $\sigma$85 CPU on a scene with about $\sigma$86M triangles and about $\sigma$87M dynamic triangles (Gajmer et al., 11 May 2026).

In x-ray and CT tomography, GRM treats each detector reading as the line integral of a continuous attenuation field along a geometric ray. Under Beer–Lambert,

$\sigma$88

and with a continuous image model

$\sigma$89

the forward operator over a ray set becomes

$\sigma$90

The contribution $\sigma$91 is the exact line integral of one shifted basis function, and for directional factorizations of $\sigma$92 it becomes a closed-form convolution of scaled one-dimensional atoms. The paper develops a generalized ray-tracing implementation for arbitrary collections of lines, including splines and box-splines with overlapping support, and keeps forward and backward operators exactly adjoint in matrix-free form. In fan-beam experiments on CT lung images, degree-$\sigma$93 box-splines report the best PSNR and SSIM at each tested grid size, including $\sigma$94 dB and $\sigma$95 at $\sigma$96. On CPU at $\sigma$97, the reported forward and backward times are $\sigma$98 s and $\sigma$99 s versus ASTRA’s $c$00 s and $c$01 s; on GPU, ASTRA is faster, with $c$02 s and $c$03 s versus $c$04 s and $c$05 s (Haouchat et al., 26 Mar 2025).

Across these domains, the reported limitations are structurally similar even when the applications differ. CeRF reports higher compute and memory footprint due to conv and RNN, LaRI reports sensitivity to textureless regions, specular and transparent materials, and thin structures, VistaRef notes failures at very long pointing distances or low hand resolution and does not specify multi-hand handling, GCRayDiffusion remains sensitive to extremely low overlap and weak image features, and vectorial GRIN tracing remains first-order accurate and may require small $c$06 for steep gradients or highly curved paths (Yang et al., 2023, Li et al., 25 Apr 2025, Li et al., 23 Jun 2026, Chen et al., 28 Mar 2025, Chávez-Islas et al., 21 Jun 2026). The future directions named in the cited works are correspondingly ray-centric: adaptive $c$07 or hard first-hit constraints, transformer-based sequence models along rays, variable numbers of layers per pixel, uncertainty modeling and multi-view fusion, integration of photometric losses into ray-conditioned reconstruction, and adaptive reduction of optical path length step size near high curvature or close boundaries (Yang et al., 2023, Li et al., 25 Apr 2025, Chen et al., 28 Mar 2025, Chávez-Islas et al., 21 Jun 2026).

A plausible implication is that GRM is most useful when the dominant ambiguity is not the existence of a signal but its placement, ordering, or transport along a known family of rays. In that regime, ray-wise structure acts as an explicit inductive bias: it can turn underdetermined volumetric integration into unimodal surface selection, convert single-view inference into layered intersection prediction, regularize unposed reconstruction through endpoint-consistent ray bundles, and replace global attention with geometric orientation cues.