Map-to-Ray Matching Strategy

• Map-to-ray matching is a strategy that models spatial maps and sensor rays using constrained zonotopes and ray-centric partitioning to precisely associate observations with physical features.
• It enhances GNSS shadow matching by computing exact shadow volumes through Minkowski sums and efficient set operations, offering robustness and computational speed.
• The approach integrates ray-centric query initialization in 3D detection, reducing redundancy and resolving perspective ambiguities to improve autonomous navigation performance.

Map-to-ray matching is a geometric-computational strategy for associating information from a spatial “map” (e.g., 3D models, feature representations) with the structure of rays defined by physical sensors or imaging geometry. This approach arises in multiple applied domains, notably in set-valued GNSS shadow matching with constrained zonotopes for urban localization, and in multi-camera 3D object detection with ray-centric query initialization in neural architectures. Core principles include explicit modeling of the map and ray geometry to achieve efficient, unambiguous assignment of observations, often improving robustness and computational scalability relative to grid-based baselines.

1. Theoretical Foundations and Problem Scope

Map-to-ray matching addresses the challenge of relating a spatial map—typically represented as a set of convex bodies or rasterized grids—to rays defined by sensor lines-of-sight, satellite ephemerides, or camera projection geometry. The task is to compute set relationships (e.g., intersection, containment, feature association) between the physical environment and these rays, using mathematical constructs that preserve key geometric properties.

In GNSS shadow matching, the objective is to infer possible receiver positions given map obstructions (buildings) and observed line-of-sight (LOS) or non-line-of-sight (NLOS) signals from satellites, via the computation of shadow volumes and their ground intersections (Bhamidipati et al., 2022). In multi-camera 3D detection, the challenge is to align neural object queries with the geometry of camera rays to ensure that distinct queries extract unique features and resolve ambiguities arising from perspective projection, as explicitly modeled in RayFormer (Chu et al., 20 Jul 2024).

2. Map Representation and Ray Modeling

2.1 Constrained Zonotopes for Set-Valued Geometry

Constrained zonotopes are a class of convex polytopes formally specified as

$$Z = \mathcal{Z}(c, G, A, b) = \left\{x = c + G\beta: \beta \in [-1,1]^m,\, A\beta = b\right\}$$

with $c$ (center), $G$ (generator matrix), and linear constraints $(A, b)$ supporting complex polytope shapes while allowing algebraic operations such as Minkowski sum and intersection.

Building models in urban GNSS applications are encoded as finite unions of constrained zonotopes. For each satellite $s_j$ and building $B_i$, a representative point $b_i$ allows definition of a shadow direction $d_{i,j} = (s_j - b_i)/\|s_j - b_i\|_2$. The shadow cast by $B_i$ in direction $d_{i,j}$ is the Minkowski sum $B_i \oplus D_{i,j}$, where $D_{i,j}$ is a zonotope describing the ray segment with sufficient span to encompass urban shadow volumes (Bhamidipati et al., 2022).

2.2 Ray-Centric Partitioning for Feature Association

In camera-based 3D detection, the “ray” abstraction corresponds to the lines connecting the ego-vehicle center to each pixel in perspective-view images. In RayFormer, the Bird’s-Eye-View (BEV) plane is tessellated into “ray–ring” sectors by quantizing polar angle $\theta$ and radius $\rho$, assigning each grid cell $(x, y)$ to a unique ray index $r$ and depth-ring $d$ using

$$r = \left\lfloor \frac{\theta}{\Delta\theta} \right\rfloor, \quad d = \left\lfloor \frac{\rho}{\Delta\rho} \right\rfloor$$

where $\Delta\theta$ and $\Delta\rho$ are quantization steps. This structure enables precise association of spatial queries and map features along specific camera rays (Chu et al., 20 Jul 2024).

3. Map-to-Ray Matching Algorithms

3.1 Zonotope Shadow Matching (ZSM) Procedure

Given building zonotopes $B=\{B_i\}$, satellites $S=\{s_j\}$, received signal strengths $R=\{r_j\}$, threshold $\gamma$, and ground regions $A_k$, ZSM executes:

• For each satellite–building pair, computes $d_{i,j}$, constructs ray segment $D_{i,j}$, forms the 3D shadow volume $\mathcal{V}_{i,j}=B_i \oplus D_{i,j}$, and computes the 2D ground shadow region $S_{i,j}$ via zonotope intersection.
• For each satellite $j$, aggregates $S_j = \bigcup_{i,k}S_{i,j,k}$ and updates the set-valued position estimate $P$ by intersection (if $r_j<\gamma$, NLOS) or set-difference (if $r_j\geq\gamma$, LOS).

$$P^{(\text{after }j)} = \begin{cases} P^{(\text{before }j)} \cap S_{j} & r_j<\gamma \ P^{(\text{before }j)} \setminus S_{j} & r_j\geq\gamma \end{cases}$$

These operations result in an exact, convex-polytope set of possible positions, robust to map discretization error and amenable to constraint propagation (Bhamidipati et al., 2022).

3.2 Ray-Centric Query Placement and Feature Sampling

RayFormer employs a radial, sparse query initialization strategy:

• The BEV is partitioned into $N_r$ rays and $N_d$ depth-rings.
• Each base query is placed at

$$(x_{r,d}, y_{r,d}, z_0) = \big( (d+\tfrac{1}{2})\Delta\rho\cos((r+\tfrac{1}{2})\Delta\theta),\; (d+\tfrac{1}{2})\Delta\rho\sin((r+\tfrac{1}{2})\Delta\theta),\; z_0\big)$$

ensuring only one query per ray-ring cell and avoiding redundant queries within the same frustum.

• Query feature extraction uses both deformable attention in BEV, and sampling of image features along vertical columns corresponding to the camera ray, with learnable offsets per depth anchor (Chu et al., 20 Jul 2024).

Pseudo-code for the key decoder layer’s ray sampling is provided in the original text. For foreground enhancement, 2D object detections are used to inject additional queries along rays intersecting the bounding boxes, and an angular cost is introduced into the Hungarian assignment process to reinforce ray exclusivity.

4. Computational Properties and Implementation

Map-to-ray matching with constrained zonotopes in ZSM exhibits

• Minkowski sums: Linear expansion in generator dimensionality, $O(n(m_1+m_2)^2)$ for polytopic operations, a one-order-of-magnitude speedup over vertex hull representations (e.g., median $5$ ms vs $70$ ms per operation).
• Intersections: Construction of larger generator and constraint matrices, with performance polynomial in combined size.
• Memory efficiency: No exhaustive ground grid discretization; memory and compute scale with the number of zonotopes (number of buildings, satellites, and ground region partitions) rather than with $1/\Delta^2$ for grid-based SM.

Ray-centric query initialization in RayFormer ensures that queries are non-redundant, allowing for sparse coverage ($135 \times 6 = 810$ base queries, with $\sim90$ extra for 2D foreground rays), substantial reduction in same-ray ambiguity, and improved computational tractability as compared to exhaustive grid search (Bhamidipati et al., 2022, Chu et al., 20 Jul 2024).

5. Comparative Evaluation and Empirical Results

Shadow Matching Approaches

Method	Offline (s)	Online (s)	Error m	Set Bound m
ZSM	$1.7\times10^{-4}$	0.39	[3.46, 16.05]	[17.87, 50.11]
SM, grid = 5 m	1524.4	1.66	[3.00, 2.00]†	[59.6, 447.5]
SM, grid = 10 m	405.3	0.08	[3.00, 7.00]†	[66.7, 455.8]

† top-3 candidates (best error).

Ray-Centric Detection

RayFormer achieves 55.5% mAP and 63.3% NDS on the nuScenes dataset, demonstrating that the explicit alignment of queries and feature sampling with camera ray geometry resolves selection ambiguity and yields robust improvements in 3D detection performance (Chu et al., 20 Jul 2024).

A plausible implication is that map-to-ray matching, when precisely implemented per the described strategies, enables both a continuum set-valued localization framework (with rigorous convex bounds) and efficient, ambiguity-free object detection pipelines, outperforming earlier grid- or frustum-based alternatives.

6. Domain-Specific Significance and Extensions

Map-to-ray matching establishes the geometric foundation for robust position and perception estimation in complex environments.

• In GNSS localization, it directly supports safety constraints and robustness by maintaining a continuum of feasible positions and propagating map-derived constraints.
• In camera-based scene understanding, it structures spatial queries and feature aggregation in line with inherent optical geometry, reducing feature redundancy and optimizing coverage.

This suggests applicability to additional domains involving ray-like sensor modalities (e.g., lidar, radar), or in multi-agent settings where relative ray geometry encodes essential visibility and communication constraints.

7. Outlook and Open Problems

While map-to-ray matching demonstrates clear empirical and computational advantages, open research questions include:

• Generalizing efficient set-valued operations (e.g., for non-convex obstacles or probabilistic rays).
• Integrating dynamic map updates or real-time object motion into the zonotope and ray-centric frameworks.
• Further reducing intersection and projection complexity for ultra-dense environments or high-frequency applications.

Continued development in this area is likely to yield advances in real-time localization, perception, and their joint optimization under uncertainty (Bhamidipati et al., 2022, Chu et al., 20 Jul 2024).