Projected Orientation Consistency Loss

• The paper introduces a loss that leverages Modified Rodrigues Parameters to project SO(3) rotations into R³, transforming non-convex optimization into a convex ℓ2 loss.
• It enforces relative rotational consistency by minimizing one-sided squared errors between predicted and ideal rotations, facilitating better convergence.
• Empirical benchmarks demonstrate faster convergence and lower angular errors in both direct parameter optimization and CNN-based orientation prediction.

The projected orientation consistency loss is a self-supervised learning objective designed to enforce relative rotational consistency in orientation estimation tasks. It enables deep networks and optimization algorithms to align predicted absolute rotations with available relative rotation measurements, while avoiding non-convexities intrinsic to optimization directly over $SO(3)$, the group of 3D rotations. This loss leverages the Modified Rodrigues Parameters (MRP) to stereographically project unit quaternions onto an open Euclidean space, $\mathbb{R}^3$, making the optimization well-behaved, convex, and free of topological pathologies present in $SO(3)$. This approach is especially effective for scenarios with only relative supervision, such as unlabeled or weakly labeled pose data, and is validated in both direct parameter optimization and end-to-end neural network training regimes (Okorn et al., 2022).

1. Mathematical Foundation: Modified Rodrigues Parameters (MRP)

Orientation in $SO(3)$ is commonly represented via unit quaternions $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$, where $\rho \in \mathbb{R}$ is the scalar part and $\nu \in \mathbb{R}^3$ the vector part. The MRP is a stereographic projection mapping the closed manifold $S^3$ to the open set $\mathbb{R}^3$: $\psi = \phi(q) = \frac{\nu}{1 + \rho}$ with inverse: $\mathbb{R}^3$0 Operating in $\mathbb{R}^3$1-space ($\mathbb{R}^3$2) removes antipodal and non-convex ambiguities inherent to $\mathbb{R}^3$3, facilitating gradient-based optimization.

2. Construction of Projected Orientation Consistency Loss

Given predicted absolute rotations $\mathbb{R}^3$4 and a measured relative rotation $\mathbb{R}^3$5 (ideally $\mathbb{R}^3$6), each predicted rotation is mapped sequentially:

• Matrix $\mathbb{R}^3$7 quaternion: $\mathbb{R}^3$8
• Quaternion $\mathbb{R}^3$9 MRP: $SO(3)$0

The relative rotation is similarly converted: $SO(3)$1. The “ideal” orientation for node $SO(3)$2 in quaternion space is: $SO(3)$3 where $SO(3)$4 denotes quaternion multiplication. Both $SO(3)$5 and its antipode $SO(3)$6 are projected to $SO(3)$7: $SO(3)$8, $SO(3)$9. The one-sided squared $SO(3)$0 errors are: $SO(3)$1 The loss for the pair $SO(3)$2 is: $SO(3)$3 The total loss is computed by summing or averaging $SO(3)$4 over all supervised edges in the neighbor graph.

The gradient with respect to $SO(3)$5 is the difference to the correct projection (choosing sign for minimal loss), optionally clipped to norm $SO(3)$6 (e.g., $SO(3)$7) to combat stereographic distortion: $SO(3)$8 Clipping: if $SO(3)$9, set $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$0. The update is $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$1, with learning rate $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$2.

3. Behavior and Theoretical Properties Compared to SO(3) Losses

$q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$3 is a compact manifold with antipodal identifications and multiple geodesics between distant points, precluding global convexity. In contrast, $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$4 is open, simply connected, and has a unique, convex squared Euclidean distance between any pair of points. Stereographic projection removes SO(3)’s wrap-around and antipodal ambiguity, except at a singular point ($q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$5) not encountered under small-motion supervision.

A 3-node cycle analysis in Appendix A of (Okorn et al., 2022) demonstrates that SO(3) averaging admits a broad family of spurious critical points, while the MRP-based loss reduces these to a single degenerate family occurring only when all $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$6 and true rotations are evenly spaced about an axis. Thus, the projected loss strongly suppresses spurious optima, producing a more favorable optimization landscape.

4. Integration Into Optimization and Deep Neural Network Training

Direct Rotation-Parameter Optimization

• Rotations are parameterized by $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$7.
• At each SGD step, one index $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$8 and neighbor $q = [\rho, \nu] \in S^3 \subset \mathbb{R}^4$9 are sampled. Gradient $\rho \in \mathbb{R}$0 is computed and clipped, followed by $\rho \in \mathbb{R}$1.
• On convergence, reconvert $\rho \in \mathbb{R}$2 via $\rho \in \mathbb{R}$3, then to $\rho \in \mathbb{R}$4.
• Typical hyperparameters: $\rho \in \mathbb{R}$5, $\rho \in \mathbb{R}$6, batch size = 1, up to $\rho \in \mathbb{R}$7 steps.

End-to-End CNN-Based Orientation Prediction

• A small “$\rho \in \mathbb{R}$8-head” is attached atop a ResNet-18 backbone; network outputs $\rho \in \mathbb{R}$9.
• Compute $\nu \in \mathbb{R}^3$0, then $\nu \in \mathbb{R}^3$1, then project: $\nu \in \mathbb{R}^3$2.
• Only $\nu \in \mathbb{R}^3$3 and $\nu \in \mathbb{R}^3$4 receive gradient; neighbors' estimates are fixed for backpropagation.
• Optimization via Adam, learning rate $\nu \in \mathbb{R}^3$5, batch size 32 (20 for Pascal3D+), 10K–80K steps. No explicit weight decay beyond standard ResNet-18 setup.

5. Empirical Results and Comparative Performance

Empirical benchmarks on (Okorn et al., 2022) highlight the projected orientation consistency loss's advantages over SO(3)- and quaternion-based losses:

Setting	Metric	MRP	Quaternion	SO(3)-averaging	4D PMG
Direct (N=100)	Steps to $\nu \in \mathbb{R}^3$65° error	≈37.5K	$\nu \in \mathbb{R}^3$7100K	$\nu \in \mathbb{R}^3$8100K	$\nu \in \mathbb{R}^3$9100K
Robustness (Direct)	% converged trials	100%	82%	94%	90%
CNN (YCB Drill)	Final mean pairwise error	3.7°	28.8°	–	123°
CNN Oracle (abs. sup.)	Final mean pairwise error	1.6°	–	–	–

In 1DSfM structure-from-motion, after 20K steps, MRP achieves lower relative and absolute angular errors than all other local-SGD methods. Across Pascal3D+ categories and ModelNet40 airplane, MRP outperforms PMG and quaternion losses by several degrees.

6. Practical Considerations, Limitations, and Theoretical Insights

Theoretical analysis shows that projecting to $S^3$0 via MRP eliminates most non-convexity in the optimization landscape, except for a degenerate case when all estimated rotations are at the origin. Clipping the gradient step size prevents distortion due to the stereographic projection’s amplification effects. While the projection singularity exists at $S^3$1, small-motion regimes effectively avoid this issue.

By enforcing pairwise relative-rotation consistency in $S^3$2-space, one obtains a squared-$S^3$3 loss with convexity and convergence properties unattainable in direct $S^3$4 loss minimization (Okorn et al., 2022). This results in more reliable, robust, and efficient training dynamics, verified both in direct optimization and deep neural network training.