Direct Object-Relative Pose Measurement

• Direct Object-Relative Pose Measurement is a set of techniques that compute 6-DoF transformations by directly comparing sensor observations without reconstructing the entire scene.
• The approach leverages latent orbit generators and group-theoretic equivariance to robustly encode pose information and handle symmetric ambiguities.
• Empirical evaluations show improved sensitivity and computational efficiency on benchmarks like EPFL Car Dataset and PASCAL3D+ using novel cyclic cross-correlation methods.

Direct Object-Relative Pose Measurement refers to the family of computational and algorithmic techniques that estimate the rigid-body relative transformation—typically 6 degrees-of-freedom (DoF) rotation and translation—between two physical objects, or between a sensor (e.g., robot, camera) and an object, without requiring indirect scene reconstruction or batch optimization. Distinct from classical absolute pose paradigms, direct object-relative pose measurement computes the transformation by comparing sensor observations in a manner that is often equivariant to group actions, robust to symmetry, and suitable for real-time applications. This article reviews the key principles, leading architectures, group-theoretic underpinnings, and evaluation protocols for such systems, with detailed reference to the orbit-based method in "Estimating Small Differences in Car-Pose from Orbits" (Kicanaoglu et al., 2018), and situates them within the broader context of object-relative perception.

1. Formal Problem Definition and Mathematical Foundation

The task is to estimate the relative pose (rotation and/or translation) between two object states, denoted as $(\mathcal{O}_1, \theta_1)$ and $(\mathcal{O}_2, \theta_2)$, from sensor observations. For a rigid object under rotation subgroup $G \subset SO(3)$ (e.g., azimuthal rotations about the $z$-axis), the goal is to measure the difference $\Delta \theta = \theta_2 - \theta_1$ directly from data. This approach is differentiated from classical schemes that predict the absolute object pose in a canonical frame and then differencing them; instead, the measurement process itself is designed to output $\Delta \theta$ (Kicanaoglu et al., 2018).

The subgroup $G$ is sampled at $K$ equispaced azimuth angles, with generator

$$g = R_z(\Delta \theta),\quad \Delta \theta = 2\pi / K$$

and group elements $g_k = g^k,\ k=0,\dots,K-1$.

A latent orbit generator is constructed so that for each input view at (unknown) angle $\theta_0$, the encoding process produces both a pose-invariant appearance code $f_{\rm id}$ and a two-dimensional pose code $f_{\rm pose}$ that transforms equivariantly under $G$. The orbit in latent space

$$\mathcal{X}_p = \left[e \cdot f_{\rm pose},\ p \cdot f_{\rm pose},\dots,\ p^{K-1} \cdot f_{\rm pose}\right] \subset \mathbb{R}^{K\times 2}$$

thus captures the cyclic SO(2) structure intrinsic to $z$-axis rotations.

2. Orbit Generator Network Architecture and Training

The orbit-based direct pose measurement framework relies on a pair of neural components: an encoder network $\mathcal{F}_{\rm enc}$ and a group-theoretic latent rotation module. The encoder ingests a single RGB input image $x \in \mathbb{R}^{64\times64\times3}$ and outputs a code $\mathbf{f}_e = [f_{\rm id}; f_{\rm pose}]$ with $f_{\rm id} \in \mathbb{R}^{128}$ (appearance, invariant) and $f_{\rm pose} \in \mathbb{R}^2$ (pose, equivariant) (Kicanaoglu et al., 2018).

To train the network to represent subgroup-equivariant structure, four objective terms are imposed:

1. Radius loss: Ensures $f_{\rm pose}$ lies on a circle of radius $c$.
2. Pairwise loss: Enforces equivariance, aligning $f_{\rm pose}(\theta_j)$ with a rotation $p^N f_{\rm pose}(\theta_i)$ for a known offset $N$.
3. Symmetry constraint: Embodies swap invariance to handle object symmetries.
4. Reconstruction loss: Ensures reconstructed images from the orbit match ground truth.

The total loss is

$$L_{\rm total} = \beta_1 L_{\rm recon} + \beta_2 L_{\rm radius} + \beta_3 L_{\rm pair}$$

The decoder component is used only in training to enforce cycle consistency; after training, only the encoder and latent-group action remain active.

3. Orbit Metric and Relative Pose Inference

Direct object-relative pose measurement is realized via orbit comparison: given two encoded latent orbits $X^1_{p}, X^2_{p} \in \mathbb{R}^{K \times 2}$, a cyclic cross-correlation is performed for all possible shifts:

$$M(\delta) = \sum_{k=0}^{K-1} \left\langle X^1_{p}[k],\ p^\delta X^2_{p}[k] \right\rangle$$

The optimal shift $\Delta \delta^* = \arg\max_\delta M(\delta)$, and the relative azimuth difference is $$\Delta \theta_{\rm rel} = \Delta \delta^* \cdot \Delta \theta$$.

If one orbit is derived from a reference view of known angle $\theta_{\rm ref}$, absolute test pose is reconstructed as $$\theta_{\rm test} = \theta_{\rm ref} + \Delta \theta_{\rm rel}\ \text{mod}\ 2\pi$$. This directed orbit metric satisfies nonnegativity, identity of indiscernibles, and triangle inequality.

4. Group-Theoretic Interpretation and Equivariance

The core architectural innovation is full group equivariance in latent space: the 2-dimensional $f_{\rm pose}$ code transforms under latent $p$ exactly as 3D rotation acts on the object. The orbit sample $X_p$ thus becomes an equivariant embedding for the subgroup $G$, supporting direct, symmetry-aware, and ambiguity-resolving relative pose measures. Comparing entire orbits (rather than single-point regressions) enables sensitivity to small differences, intermediate and opposite poses, and robustness to symmetric ambiguities (Kicanaoglu et al., 2018).

This paradigm subsumes classical group actions and extends direct pose measurement into a general frame for $G \leq SO(3)$, mapping subgroup rotations to cyclic structure in $\mathbb{R}^2$.

5. Practical Performance and Evaluation Metrics

In the central experimental setup, the method is trained first on synthetic car models rendered at $10^\circ$ azimuth increments with random elevation and background, and finetuned on real-world crops for domain transfer (Kicanaoglu et al., 2018). In testing:

• EPFL Car Dataset: Accuracy-36 (discrete azimuth binning, correct if within same of 36 bins):
  • Orbit-based method: 54.0% vs prior best ≈ 52.1%
• PASCAL3D+ Cars (AVP-24):
  • Orbit-based: 28.3% vs prior ≈ 25.5%
• Error-type analysis (nearby/opposite/other):
  • Orbit-based: 9.6% / 8.9% / 35%
  • Best prior: 12% / 5% / 18%
  • Notably achieves the lowest nearby-pose error, indicating heightened sensitivity to local differences.

The inference pipeline is direct and computationally efficient: it entails a forward pass for each image to encode $f_{\rm pose}$, followed by $K\times K$ cyclic dot-products for orbit alignment.

6. Algorithmic Workflow

The core algorithm for direct object-relative pose measurement in the orbit-based system is:

f_id^r, f_pose^r = Encoder(I_ref) f_id^t, f_pose^t = Encoder(I_test) Xr = [p^k @ f_pose^r for k in range(K)] Xt = [p^k @ f_pose^t for k in range(K)] M = [ sum( dot(Xr[k], p^δ @ Xt[k]) for k in range(K) ) for δ in range(K) ] Δδ = argmax(M) Δθ = Δδ * Δθ θ_test = θ_ref + Δθ mod 2π

(Kicanaoglu et al., 2018)

This direct relative pose comparison—operating entirely in the latent, equivariant code space—eliminates the need for algebraic inversion or multiple hypotheses per symmetry and avoids error propagation typical of absolute-pose differencing.

7. Significance, Robustness, and Broader Implications

Direct object-relative pose measurement, as instantiated in the orbit method, advances the field through several axes:

• Sensitivity: The equivariant orbit architecture sharply resolves small pose differences, outperforming prior pointwise or keypoint-augmented regressions for both nearby and ambiguous poses.
• Symmetry Handling: Object symmetries are naturally embedded via latent group actions and symmetry constraints, allowing the method to learn distributions over poses rather than forcibly disambiguated point targets.
• Computational Efficiency: Architectures are lightweight (encoder + group action), require no iterative matching, and generalize rapidly from synthetic to real domains.
• Theoretical Guarantees: The group-theoretic formulation ensures the invariances and properties needed for well-behaved relative pose metrics (triangle inequality, nonnegativity, etc.).

These properties make direct pose measurement applicable to robotics, manipulation, navigation, and vision systems where precision, ambiguity resolution, and efficiency are paramount.

---

Key reference: Kicanaoglu, C., Erdem, A., & Erdem, E. "Estimating Small Differences in Car-Pose from Orbits" (Kicanaoglu et al., 2018).