---
title: 'PlanaReLoc: Planar-Based 6-DoF Relocalization'
url: https://www.emergentmind.com/topics/planareloc
type: topic
---

# PlanaReLoc: Planar-Based 6-DoF Relocalization

PlanaReLoc is a structure-based camera relocalization method that formulates query-to-map association around **planar primitives** rather than 2D–3D point correspondences. Introduced in “PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching” [2603.20818], it targets lightweight **6-DoF** relocalization in structured indoor environments using a **3D planar map** that can remain entirely untextured. Its central premise is that walls, floors, tabletops, doors, cabinets, and related planar surfaces are both geometrically fundamental and practically compact, making them suitable as the basic entities for cross-modal matching between a query RGB image and a geometry-only map.

## 1. Problem setting and plane-centric formulation

PlanaReLoc addresses **camera relocalization**: given a query image \(q\), estimate the camera pose relative to a known scene map as
\[
P := [R \mid t] \in \mathrm{SE}(3),
\]
with \(R \in \mathrm{SO}(3)\) and \(t \in \mathbb{R}^3\) [2603.20818]. The method departs from the dominant point-centric paradigm. Instead of recovering 2D–3D point correspondences and solving a PnP-style problem, it establishes correspondences between **query-side planar regions** and **map-side 3D planar primitives**.

The motivation is explicitly geometric and representational. Planes are treated as canonical entities in projective geometry, but also as **region-based representations** that aggregate evidence over extended image support. This region-based character is intended to improve robustness in weak-texture indoor scenes, partial occlusion, and repeated small-scale detail, while also enabling relocalization from a map that does not store realistic texture or color [2603.20818]. The method is therefore positioned against several alternative regimes: SfM point maps, image-to-render matching against textured maps, and image-to-point-cloud registration across modalities.

The target domain is **structured indoor environments**. The paper emphasizes that such scenes are dominated by planar surfaces and that a planar map is substantially lighter than dense textured meshes, dense point clouds, or large keypoint maps. A practical implication is that PlanaReLoc is designed for settings where memory footprint and map appearance requirements matter as much as relocalization accuracy.

## 2. Query and map primitives

The planar map is denoted
\[
\mathcal{M} = \{m_i\}_{i=1}^{N_m},
\]
where each map primitive \(m_i\) carries plane parameters \(\pi_i^m\) and a bounded spatial shape \(s_i^m\) [2603.20818]. Planes are parameterized as
\[
\pi := [n^\top, d]^\top,
\]
with \(n \in \mathbb{R}^3\) the plane normal and \(d\) the offset. On the map side, the normal orientation is kept consistent with the original surface normal.

On the query side, the image is converted into a set of recovered planar primitives
\[
\mathcal{Q} = \{q_i\}_{i=1}^{N_q},
\]
each with predicted metric plane parameters \(\pi_i^q\) and a binary 2D mask \(s_i^q\) [2603.20818]. Query planes are not obtained from a semantically heavy detector; instead, the paper uses **MoGe-2** for monocular metric geometry estimation followed by **sequential RANSAC plane fitting**. This front-end is described as purely geometric and plug-and-play.

Several implementation details delimit the primitive extraction regime. The input image is resized to \(640 \times 480\); the feature map is \(80 \times 60\); fitting operates on downsampled depth; the inlier threshold is a point-to-plane residual below 10 cm and a normal-similarity dot product above 0.9; and extraction stops after 16 planes or when the inlier count falls below 1% of pixels [2603.20818]. The result is a compact query representation consisting of at most 16 planar regions.

The query embeddings are formed by pooling image features over each recovered segment mask. Specifically, the image is patchified by a pretrained encoder, masks are resized to feature-map resolution, and **average pooling** over each segment produces query-plane embeddings
\[
\{e_i^q \in \mathbb{R}^c\}_{i=1}^{N_q},
\]
with embedding dimension \(c = 384\) [2603.20818]. On the map side, embeddings
\[
\{e_i^m \in \mathbb{R}^c\}_{i=1}^{N_m}
\]
combine two PointNet-based components: an **object encoder** over centralized per-plane point clouds and a **scene encoder** that captures pose-aware spatial context in the whole map. Each primitive is represented with \(L = 1024\) sampled points, and the two embeddings are fused by a learnable \(\alpha\)-weighted sum [2603.20818].

This two-branch map encoding is structurally important. The object encoder captures local plane shape, while the scene encoder disambiguates repeated local geometry by encoding scene context. The paper’s interpretation is that neither local shape alone nor scene context alone is sufficient for reliable planar matching in indoor scenes with repeated walls or furniture panels.

## 3. Cross-modal planar matching

PlanaReLoc uses a transformer matcher to associate query-plane embeddings and map-plane embeddings inside a shared cross-modal embedding space [2603.20818]. The architecture is a stack of \(N = 4\) identical transformer layers, each with one self-attention unit and one cross-attention unit, and each unit uses 4 attention heads. The model is inspired more by assignment learning in the style of SuperGlue and LightGlue than by pure contrastive embedding objectives.

A key geometric design is the injection of relative normal information into self-attention. The paper writes the modified self-attention score as
\[
a_{ij} = q_i^\top\, R(n_j - n_i)\, k_j,
\]
with a rotary positional encoding
\[
R(\cdot) = \left( \begin{smallmatrix} R(b_1^\top \cdot) & & \\ & \ddots & \\ & & R(b_{c/2}^\top \cdot) \end{smallmatrix} \right), \quad
R(\theta) = \left( \begin{smallmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{smallmatrix} \right),
\]
where each \(b_k \in \mathbb{R}^3\) is learnable [2603.20818]. The operational role of this term is to expose relative plane-orientation cues directly inside attention.

The matcher predicts a soft assignment matrix
\[
S \in \mathbb{R}^{N_q \times N_m},
\]
with entries combining pairwise similarity and predicted matchability:
\[
S_{ij} = \sigma_i^q \sigma_j^m\, \softmax_{k \in [1,N_q]}(A_{kj})_i\, \softmax_{k \in [1,N_m]}(A_{ik})_j.
\]
Here,
\[
A_{ij} = \linear(e_i^q) \cdot \linear(e_j^m),
\]
and the per-primitive matchability is
\[
\sigma_i = \sigmoid(\linear(e_i)) \in [0,1]
\]
[2603.20818]. At inference time, correspondences are accepted only if they pass a confidence threshold \(\tau\) and satisfy the **Mutual Nearest Neighbor** criterion.

Training labels are constructed from ground-truth camera pose by projecting map primitives into the query image and selecting, for each recovered query primitive, the map primitive with the highest projected mask IoU [2603.20818]. Importantly, the supervision is **not bipartite**: one map primitive may supervise multiple query primitives, reflecting over-segmentation in the query due to occlusion, truncation, or imperfect plane extraction. Unmatchable query and map primitives are collected into \(U_q\) and \(U_m\) based on an IoU threshold \(\delta\).

The matching objective is
\[
\begin{split}
L = - \bigg( & \frac{1}{|M^*|}\sum_{(i,j)\in M^*} \log S_{ij} + \frac{1}{2|U_q|}\sum_{i\in U_q} \log (1 - \sigma_i^q) \\
& + \frac{1}{2|U_m|}\sum_{j\in U_m} \log (1 - \sigma_j^m) \bigg),
\end{split}
\]
and is applied at every transformer layer for deep supervision [2603.20818].

## 4. Pose recovery from matched planes

Once plane correspondences are established, PlanaReLoc solves pose directly from plane geometry. If the camera pose maps camera-space points to map-space points as \(x^m = P x^q\), plane parameters transform by
\[
\pi^m \sim P^{-\top}\pi^q \sim
\begin{pmatrix} R & t \\ 0^\top & 1 \end{pmatrix}^{-\top}\pi^q
\sim
\begin{pmatrix} R & 0 \\ - t^\top R & 1 \end{pmatrix}\pi^q.
\tag{2}
\]
From this, the paper derives
\[
n^m = R n^q,
\]
and
\[
d^m = d^q - t^\top n^q = d^q - t^\top n^m.
\tag{3}
\]
Thus, matched normals constrain rotation, while offsets constrain translation once rotation is known [2603.20818].

Rotation is estimated robustly with **RANSAC**. Two non-parallel matched plane pairs form the minimal sample used to hypothesize \(R\); the best inlier set \(\mathcal{I}\) is then used to recompute an initial rotation \(R_0\) via the **Kabsch algorithm** [2603.20818]. Because monocular plane recovery introduces scale error into query plane offsets, translation is solved jointly with a global scale factor \(s\):
\[
t^*, s^* = \argmin_{t, s} \sum_{(i,j)\in\mathcal{I}} w_i \left(t^\top n^m_j - d^m_j + s\, d^q_i\right)^2.
\tag{4}
\]
The weight \(w_i\) is proportional to the 2D segment size of query primitive \(q_i\), reflecting the assumption that larger recovered planes are more reliable [2603.20818].

The paper rewrites this as a weighted linear least-squares problem. For each inlier correspondence \((i,j)\),
\[
a := [(n^m_j)^\top,\; d^q_i], \qquad b := d^m_j,
\]
which yields
\[
A x \approx b, \qquad x := [t^\top, s]^\top \in \mathbb{R}^4,
\]
and
\[
x^* = \argmin_x \|W(Ax - b)\|_2^2.
\tag{5}
\]
Translation requires at least **three non-parallel** correspondences, and the reported degeneracy rate is below 2% empirically [2603.20818]. When degeneracy occurs, the method falls back to a heuristic coarse pose estimated from predicted correspondences.

## 5. Depth-based refinement and empirical performance

PlanaReLoc includes a post-estimation refinement stage that jointly optimizes pose and per-query-plane offset corrections against a rendered depth map from the planar map [2603.20818]. The initial pose is denoted \(P_0\), and the refined pose is
\[
P^* = T^* \times P_0,
\]
where \(T\) is a relative transformation and \(\{\lambda_i\}\) are per-plane offset seeds. For each query primitive \(q_i\), the warped depth segment is computed by
\[
\hat{u}_i[p], \hat{z}_i[p] = \Pi \left( T\, \Pi^{-1}(p, \lambda_i D_i[p]) \right),
\tag{6}
\]
and the per-primitive residual is
\[
r(\lambda_i, T; q_i, D^r) =
\frac{1}{|s_i^q|} \sum_{p \in s_i^q}
\left( D^r[\hat{u}_i[p]] - \hat{z}_i[p] \right)^2.
\tag{7}
\]
The aggregate refinement objective is
\[
T^*, \{\lambda_i^*\} =
\argmin_{T, \{\lambda_i\}}
\frac{1}{N_q} \sum_{(q_i,\lambda_i)} r(\lambda_i, T; q_i, D^r).
\tag{8}
\]
Optimization uses Adam for 200 iterations, with \(T\) parameterized as a differentiable 6D Lie algebra variable via LieTorch, learning rate \(10^{-3}\) for \(T\), learning rate \(10^{-4}\) for \(\lambda_i\), and 4096 sampled pixels per iteration [2603.20818].

The reported relocalization results on **ScanNet** and **12Scenes** summarize both the base solver and the effect of refinement.

| Dataset | Variant | Key reported performance |
|---|---|---|
| ScanNet | Without refinement | Mean rot err 17.3, median rot err 3.9, mean trans err 0.65, median trans err 0.27, recalls 37.1 / 69.8 / 79.8, runtime 0.059854 s/query |
| ScanNet | Full PlanaReLoc | Mean rot err 17.2, median rot err 3.8, mean trans err 0.60, median trans err 0.20, recalls 48.5 / 73.1 / 81.8, runtime 0.544342 s/query |
| 12Scenes | Without refinement | Mean rot err 4.8, mean trans err 0.28, recalls 34.9 / 66.7 / 79.9 |
| 12Scenes | Full PlanaReLoc | Mean rot err 4.7, mean trans err 0.19, recalls 50.6 / 70.8 / 80.6 |

The paper also reports explicit matching metrics. On **ScanNet**, PlanaReLoc reaches precision **67.6**, recall **61.3**, F-score **64.3**, and AP **91.8**. On **12Scenes**, it reaches precision **63.9**, recall **54.2**, F-score **58.6**, and AP **87.8** [2603.20818]. These results are used to support the claim that planar primitives are effective for **cross-modal structural matching** between an RGB query and an untextured planar map.

Ablation studies indicate that several components are not incidental. Removing either the **object encoder** or the **scene encoder** degrades performance, as does removing the normal-based positional embedding. Robust estimation with **RANSAC** is reported as crucial, and explicit optimization of the global monocular scale improves pose accuracy [2603.20818]. The default query front-end, **MoGe-2 + RANSAC**, also gives the best reported speed/accuracy tradeoff among tested plane-recovery alternatives.

## 6. Map compactness, comparative position, and limitations

A major practical feature of PlanaReLoc is map compactness. The appendix reports that simplified planar maps average **154.3 KiB**, which is only **3.2%** of the colored map size [2603.20818]. This is central to the method’s deployment rationale: relocalization is performed without textured or colored maps, without pose priors, and without per-scene training.

This places PlanaReLoc in a distinct part of the localization design space. Unlike **LaLaLoc**, which localizes a single RGB panorama to a floor plan by learning a shared latent layout space and estimates a **2-DoF** planar pose in previously unvisited indoor scenes [2104.09169], PlanaReLoc solves **6-DoF** relocalization against a **3D planar map**. Unlike **UnLoc**, which performs sequential floorplan localization in \(\mathrm{SE}(2)\) using uncertain 1D floorplan-depth signatures and histogram filtering [2509.11301], PlanaReLoc is not a floorplan-localization method and does not operate over occupancy-grid pose volumes. It is also distinct from **PlaneRecTR++**, which learns joint plane reconstruction and **relative** camera pose between two images rather than **absolute** query-to-map relocalization [2307.13756].

The paper identifies several limitations. The dominant bottleneck is **monocular plane recovery**: if the front-end produces poor plane segments or inaccurate plane geometry, matching and pose estimation can fail [2603.20818]. Highly repetitive structures remain problematic; the appendix includes a case in which repeated planar layouts allow RANSAC to preserve an incorrect pose despite several correct matches. Performance also degrades in scenes with too few informative planes, in large multi-room environments with increasing structural ambiguity, and beyond the indoor structured regime for which the method was designed. The authors explicitly note that outdoor generalization is not established [2603.20818].

Within those bounds, PlanaReLoc’s main technical significance lies in its reformulation of relocalization as **region-based structure matching over planar primitives**. The method combines a query-side geometric plane recovery front-end, a map-side geometric embedding for untextured planes, a transformer assignment model with explicit matchability, a closed-form plane-based pose solver augmented by global scale compensation, and a depth-render refinement stage. This architecture suggests that, in structured indoor scenes, planar primitives can function not merely as auxiliary cues but as the primary entities for cross-modal relocalization [2603.20818].

Source: https://www.emergentmind.com/topics/planareloc