Anchored Landmark Parameterizations

• Anchored landmark parameterizations are methods that express variables relative to anchors, improving local linearity, identifiability, and numerical conditioning.
• They are applied in domains like visual relocalization, facial landmark detection, VINS, SLAM, and sensor network localization to simplify complex global estimation problems.
• Implementations range from hard-anchor selection to soft-anchor aggregation, demonstrating significant performance gains and consistent improvements across various applications.

Searching arXiv for relevant papers on anchored landmark parameterizations and adjacent usages. Anchored landmark parameterizations are a family of representations in which geometric, dynamical, or probabilistic variables are expressed relative to discrete anchors, landmarks, or anchor states rather than directly in a single global coordinate frame. Across visual relocalization, visual–inertial navigation, facial landmark localization, object-level SLAM, stochastic landmark geometry, sensor network localization, and few-step generative flow maps, the recurring construction is to decompose prediction into an anchor-selection or anchor-conditioning stage and a local representation stage defined with respect to that anchor. In the visual relocalization formulation of "Improved Visual Relocalization by Discovering Anchor Points" (Saha et al., 2018), for example, horizontal camera position is parameterized as offsets from route anchors while vertical position and orientation remain globally parameterized. In the VINS formulation of "Observability and Consistency Analysis for Visual-Inertial Navigation with Anchored Feature Parameterizations" (Cohen et al., 17 Jun 2026), landmarks are represented relative to an anchor camera/IMU pose. In "AnchorFace: An Anchor-based Facial Landmark Detector Across Large Poses" (Xu et al., 2020), 2D facial landmarks are represented as anchor-template coordinates plus regressed offsets. Although these works arise in different subfields, they share a common principle: replace a difficult global estimation problem by a structured local parameterization conditioned on one or more anchors.

1. Conceptual scope and recurring structure

An anchored landmark parameterization replaces direct global regression or direct global state storage by a representation of the form “anchor plus local coordinates,” “anchor frame plus local feature parameters,” or “anchor-conditioned stochastic bridge.” In the most explicit camera-pose case, a standard PoseNet-style method regresses the global pose directly as

$f_\theta(I) \approx (x, y, z, P) \in \mathbb{R}^6,$

whereas the anchor-based alternative in (Saha et al., 2018) regresses anchor confidences, relative horizontal offsets, and global $z$ and orientation.

The same structural idea appears in several distinct forms. In VINS, a landmark is not stored as a global Cartesian coordinate but relative to an anchoring camera frame $\mathcal{F}_{c_f}$, with the corresponding anchor state carried by the IMU pose at feature initialization (Cohen et al., 17 Jun 2026). In AnchorFace, a landmark set is not regressed directly in image coordinates but as

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$

with a learned confidence over anchor templates and spatial anchor points (Xu et al., 2020). In sensor localization, unknown points are parameterized in the MDS coordinate system induced by physical anchors and then mapped back by a Procrustes transform (Ouyang et al., 14 Sep 2025). In flow-map RL, a deterministic long-range map is stochasticized through an anchor time $\Tau$ and an anchor state $x_{\text{anc}}$, followed by conditional resampling from $p_{r\mid \Tau}$ (Li et al., 1 Jul 2026).

This suggests a general taxonomy. Some anchored parameterizations use discrete spatial anchors along a route or image lattice (Saha et al., 2018, Xu et al., 2020). Others use anchoring poses or frames that define local coordinates for landmarks (Cohen et al., 17 Jun 2026). Others use intrinsic manifold anchors that remove non-uniqueness in object representations, as with $\mathrm{SPD}(3)\times\mathbb{R}^3$ for object landmarks (Hu et al., 2022). Still others use ambient landmark spaces and then restrict to anchored submanifolds obtained by fixing some landmark coordinates (Habermann et al., 1 Jun 2026). Despite these differences, the common objective is to improve identifiability, local linearity, numerical conditioning, or statistical consistency by choosing a representation aligned with the structure of the problem.

2. Anchor-plus-offset representations in pose and keypoint estimation

In visual relocalization, (Saha et al., 2018) defines anchor points $A_1,\dots,A_N$ uniformly across the route map by selecting every $k$-th training frame as an anchor. For an image with ground-truth pose $z$0, the relative horizontal offsets with respect to anchor $z$1 are

$z$2

The network predicts per-anchor confidences $z$3, per-anchor offsets $z$4, and global $z$5. At inference time, the relevant anchor is typically

$z$6

and the global horizontal position is reconstructed as

$z$7

The representation is only partially anchored, since $z$8 and orientation remain globally regressed (Saha et al., 2018).

In facial landmark localization, AnchorFace uses a related but denser construction. A finite set of anchor templates $z$9 is defined by a spatial anchor point $\mathcal{F}_{c_f}$0 and a pose prototype $\mathcal{F}_{c_f}$1. For each anchor template, the network predicts landmark offsets $\mathcal{F}_{c_f}$2 and a confidence $\mathcal{F}_{c_f}$3, so that each anchor induces a candidate prediction

$\mathcal{F}_{c_f}$4

Final landmarks are obtained by confidence-weighted aggregation after thresholding with $\mathcal{F}_{c_f}$5: $\mathcal{F}_{c_f}$6 Here the parameterization is not tied to a single selected anchor but to a mixture over many anchor hypotheses (Xu et al., 2020).

The two formulations differ in how they use anchors at inference. The relocalization system appears to use only top-1 anchor selection (Saha et al., 2018), whereas AnchorFace explicitly aggregates over many anchors (Xu et al., 2020). This suggests two operational regimes of anchored parameterization: a hard-anchor regime, where an anchor indexes a local coordinate chart, and a soft-anchor regime, where anchors provide a set of local experts combined by confidences.

3. Landmark states relative to anchor frames in VINS and SLAM

In VINS, anchored landmark parameterization refers to representing a 3D feature relative to a specific anchor pose rather than in a global frame. The paper (Cohen et al., 17 Jun 2026) considers a global frame $\mathcal{F}_{c_f}$7, an IMU frame $\mathcal{F}_{c_f}$8 at anchoring time $\mathcal{F}_{c_f}$9, and a camera frame $$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$0 rigidly attached to the IMU. The anchored landmark coordinates are defined in the camera frame by

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$1

and the global landmark position is

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$2

The anchored inverse-depth parameterization used in the paper is

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$3

The central analytical result is that, for anchored landmark parameterizations of the form above, the unobservable subspace does not depend on the landmark states. In the notation of (Cohen et al., 17 Jun 2026), the landmark blocks of the nullspace satisfy

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$4

By contrast, global feature parameterization yields landmark-dependent nullspace blocks

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$5

This independence from landmark linearization points is the basis for the improved consistency properties reported in (Cohen et al., 17 Jun 2026).

Object-level SLAM introduces a different but related anchoring problem. Conventional object parameterizations on $$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$6 are not globally consistent because the same ellipsoid can be represented by different combinations of rotation and axis lengths. The representation proposed in (Hu et al., 2022) replaces separate pose and scale variables by a single symmetric positive-definite matrix $$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$7 together with translation $$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$8, so the object landmark state becomes

$$\text{landmarks} \approx \text{anchor template} + \text{offsets},$$9

The dual quadric is written as

$\Tau$0

Here the “anchor” is not a measurement frame but a globally consistent intrinsic representation of the object’s shape and orientation. This removes the Rot–Scale ambiguity that arises when the same abstract object can be represented by a 90° frame rotation and a swap of length and width (Hu et al., 2022).

4. Loss design, confidence weighting, and anchor discovery

A characteristic feature of anchored parameterizations is that anchor selection is often latent rather than supervised. In (Saha et al., 2018), the model does not require labels for which anchor is relevant. Instead, it uses a confidence-weighted offset loss

$\Tau$1

If anchor $\Tau$2 is irrelevant, the network can drive $\Tau$3 toward 0; if it is relevant, the model is encouraged to assign it high confidence and accurate offsets. An optional nearest-anchor cross-entropy term

$\Tau$4

can be added, but the paper reports that without this cross-entropy term, performance is often better, especially in Cambridge Landmarks, where the variant without CE performs better in 5 of 6 scenes (Saha et al., 2018). The total loss is

$\Tau$5

The “discovered anchor” variant sets $\Tau$6 (Saha et al., 2018).

AnchorFace uses a different supervision strategy. Every anchor receives a regression target and a soft confidence target derived from template proximity to the ground-truth shape. If $\Tau$7 and $\Tau$8 are flattened landmark vectors for the anchor template and the ground truth, respectively, then

$\Tau$9

with $x_{\text{anc}}$0 (Xu et al., 2020). Regression is then weighted by this confidence target,

$x_{\text{anc}}$1

and the confidence branch is trained by a binary cross-entropy-like loss, with total loss

$x_{\text{anc}}$2

using $x_{\text{anc}}$3 (Xu et al., 2020).

These two designs illustrate distinct meanings of “anchor confidence.” In (Saha et al., 2018), confidence acts as a latent selector that emerges from minimizing weighted regression error. In (Xu et al., 2020), confidence approximates template–ground-truth proximity and is then used for aggregation. This suggests that anchored parameterizations can either discover anchors endogenously from reconstruction quality or learn anchor weights from explicit geometry-derived supervisory signals.

5. Mathematical and geometric formulations beyond vision regression

Anchored landmark parameterizations also arise as coordinate systems on geometric spaces. In the landmark-space analysis of (Habermann et al., 1 Jun 2026), the basic configuration manifold is

$x_{\text{anc}}$4

an open subset of $x_{\text{anc}}$5. An anchored configuration space is obtained by fixing some landmark coordinates, for example

$x_{\text{anc}}$6

which is a submanifold of codimension $x_{\text{anc}}$7. The ambient Riemannian cometric is induced by a kernel

$x_{\text{anc}}$8

and under translation and rotation invariance,

$x_{\text{anc}}$9

Anchoring corresponds to restricting this kernel-induced structure to a submanifold where some points are frozen (Habermann et al., 1 Jun 2026). In this setting, anchoring is not primarily an estimation device but a geometric constraint on the parameter space itself.

The same paper establishes stochastic completeness results for such landmark spaces. For Sobolev/Matérn kernels with

$p_{r\mid \Tau}$0

the threshold

$p_{r\mid \Tau}$1

guarantees stochastic completeness for all $p_{r\mid \Tau}$2 (Habermann et al., 1 Jun 2026). The paper argues that anchored spaces inherit the relevant completeness, separation, and volume-growth properties from the ambient landmark space when the induced metric is restricted compatibly.

In sensor network localization, (Ouyang et al., 14 Sep 2025) formulates an anchored parameterization in an MDS coordinate system built from physical anchors $p_{r\mid \Tau}$3. After classical MDS on the anchor distance matrix $p_{r\mid \Tau}$4, the anchor coordinates in MDS space are

$p_{r\mid \Tau}$5

For a new point with squared distances $p_{r\mid \Tau}$6 to anchors, LMDS computes

$p_{r\mid \Tau}$7

A Procrustes map then returns coordinates in the physical anchor frame: $p_{r\mid \Tau}$8 where $p_{r\mid \Tau}$9 is the anchor centroid and $\mathrm{SPD}(3)\times\mathbb{R}^3$0 is determined from anchor geometry alone (Ouyang et al., 14 Sep 2025). Here the anchor set fixes translation and rotation ambiguities, and the unknown point is parameterized by its coordinates in the anchor-induced MDS system.

6. Empirical behavior across domains

The empirical consequences of anchored parameterizations depend on the application but display recurring patterns.

In visual relocalization, (Saha et al., 2018) reports improved median errors over PoseNet with geometric reprojection loss using the same feature extractor. On the Cambridge Street scene, PoseNet with geometric reprojection gives $\mathrm{SPD}(3)\times\mathbb{R}^3$1 m and $\mathrm{SPD}(3)\times\mathbb{R}^3$2, the anchor-based GoogLeNet model without CE gives $\mathrm{SPD}(3)\times\mathbb{R}^3$3 m and $\mathrm{SPD}(3)\times\mathbb{R}^3$4, and the best DenseNet anchor-based version without CE gives $\mathrm{SPD}(3)\times\mathbb{R}^3$5 m and $\mathrm{SPD}(3)\times\mathbb{R}^3$6. The translation improvement on Street is therefore more than $\mathrm{SPD}(3)\times\mathbb{R}^3$7 m (Saha et al., 2018). The same paper reports that on 7 Scenes the anchor-based method achieves $\mathrm{SPD}(3)\times\mathbb{R}^3$8 m median translation on all scenes and outperforms PoseNet variants.

The effect is not reducible to backbone choice alone. On Cambridge, a DenseNet direct regressor gives $\mathrm{SPD}(3)\times\mathbb{R}^3$9 m and accuracy $A_1,\dots,A_N$0 on Shop Facade, whereas the anchor-based DenseNet gives $A_1,\dots,A_N$1 m and accuracy $A_1,\dots,A_N$2; on King’s College, direct regression gives $A_1,\dots,A_N$3 m and $A_1,\dots,A_N$4, while the anchored version gives $A_1,\dots,A_N$5 m and $A_1,\dots,A_N$6 (Saha et al., 2018).

In facial landmark localization, AnchorFace outperforms a direct-regression baseline particularly at larger yaw. On AFLW, the baseline yields NME $A_1,\dots,A_N$7 on Full and $A_1,\dots,A_N$8 on Heavy, while AnchorFace yields $A_1,\dots,A_N$9 on Full and $k$0 on Heavy (Xu et al., 2020). Template count matters: KMeans-24 outperforms KMeans-3 and KMeans-48 on AFLW, with $k$1 NME on Full versus $k$2 and $k$3, respectively (Xu et al., 2020). Weighted aggregation also outperforms Argmax and Mean; on Heavy, Weighted gives $k$4, Argmax $k$5, and Mean $k$6 (Xu et al., 2020).

In VINS, (Cohen et al., 17 Jun 2026) reports that all estimators employing anchored feature parameterizations exhibit improved consistency properties compared to algorithms that estimate features in a global reference frame, especially when feature initialization may be poor. On TUM-VI room sequences, the monocular average ATE for Std-G3D is $k$7, whereas Std-AID gives $k$8; FEJ-AID gives $k$9; RI-AID gives $z$00 (Cohen et al., 17 Jun 2026). In stereo, Std-G3D gives $z$01, while Std-AID gives $z$02 (Cohen et al., 17 Jun 2026).

In object-level SLAM, (Hu et al., 2022) reports that the proposed $z$03 representation improves mapping accuracy by $z$04 on average using the same front-end data. On real datasets, Multi-SLAM-SPD achieves average IoU $z$05 versus $z$06 for Multi-SLAM-RTS and $z$07 for Quadric-SLAM, and average orientation error $z$08 versus $z$09 and $z$10, respectively (Hu et al., 2022). In simulation, the $z$11-based method converges $z$12 faster on average than Rot–Trans–Scale (Hu et al., 2022).

These results suggest a common empirical pattern: anchoring tends to help most when the original global parameterization is highly nonlinear, ambiguous, or sensitive to poor initialization.

7. Limitations, controversies, and open directions

Anchored landmark parameterizations do not remove all difficulties, and several limitations recur across the literature.

In visual relocalization, the anchoring is only partial. The paper (Saha et al., 2018) states that regressing relative offsets for $z$13 and orientation made learning harder and performed worse, so only $z$14 are anchored. Performance depends on anchor spacing and loss weights $z$15, and the paper notes that too many anchors or too few anchors hurt performance. The method also appears to use only top-1 anchor selection at inference, leaving probabilistic multi-anchor combination unexplored (Saha et al., 2018).

In AnchorFace, template design introduces its own discretization trade-offs. Too few templates under-cover pose variation, whereas too many can add redundancy and degrade performance, as in the comparison between KMeans-24 and KMeans-48 (Xu et al., 2020). The method handles self-occlusion implicitly rather than by explicit visibility modeling.

In VINS, anchored feature parameterizations remove landmark dependence from the unobservable subspace but do not remove navigation-state dependence. The paper (Cohen et al., 17 Jun 2026) therefore still proposes FEJ and RI-EKF variants to handle remaining inconsistency due to navigation-state linearization. Anchoring alone improves consistency, but it is not a complete solution.

In object-level SLAM, the $z$16 approach models objects as ellipsoids or circumscribed cubes rather than full shapes, and the experiments assume correct front-end data association (Hu et al., 2022). The representation removes artificial singularities but not intrinsic ambiguities of symmetric objects.

In stochastic landmark geometry, anchored spaces inherit the kernel-dependent thresholds of the full landmark manifold. The paper (Habermann et al., 1 Jun 2026) emphasizes that adding anchors does not change the critical regularity; if $z$17, collisions and stochastic incompleteness can occur even if some landmarks are fixed.

Several papers explicitly point toward extensions. The relocalization work suggests learned anchor selection, full $z$18 anchor-based parameterization, soft combination over multiple anchors, SLAM integration, hierarchical anchors, and uncertainty estimation (Saha et al., 2018). The sensor localization analysis suggests tuning a family of objectives balancing length and angle preservation, with LS emerging as a regularized LMDS solution (Ouyang et al., 14 Sep 2025). The flow-map RL paper suggests that anchor times and anchor states can function as a control mesh over stochastic trajectories without changing the original deterministic model parameterization (Li et al., 1 Jul 2026).

A plausible implication is that anchored landmark parameterizations are best understood not as a single technique but as a design pattern. The anchor may be a spatial landmark, a camera pose, a prototype template, an object-intrinsic manifold coordinate, an MDS basis, or a future waypoint in time–state space. What unifies these constructions is the decision to encode structure relative to anchors that make the remaining prediction or optimization local, better conditioned, or more faithful to the geometry of the underlying problem.