Absolute Orientation Encoding for Camera Control

• Absolute Orientation Encoding is a gravity-referenced representation that makes camera pitch and roll explicit by attaching per-token latitude and up signals.
• It is integrated with Relative Ray Encoding and image-space RoPE, utilizing a lightweight adapter with less than 1% trainable parameters to enhance video generation.
• Experimental ablations show that AOE significantly reduces pitch and roll errors while preserving lens fidelity and overall video quality in UCPE.

Searching arXiv for the primary paper and nearby camera-encoding context. Absolute Orientation Encoding (AOE) is a camera-geometry representation introduced in UCPE for camera-controlled video generation. It addresses an ambiguity that remains when camera motion is encoded only relatively: although yaw can be controlled frame-to-frame through 6-DoF pose encoding, the camera’s global tilt with respect to gravity remains unspecified. AOE resolves that ambiguity by attaching to each pixel or token two gravity-referenced signals—a latitude map and an up map—so that pitch and roll become explicit and the initial camera orientation is fully controllable. In UCPE, AOE is not used in isolation; it is combined with Relative Ray Encoding and image-space RoPE, then injected into a pretrained video Diffusion Transformer through a lightweight spatial attention adapter with less than $1\%$ trainable parameters (Zhang et al., 8 Dec 2025).

1. Definition and representational role

In the UCPE formulation, a full $SE(3)$ extrinsic pose consists of a 3-D translation and a 3-D rotation. For most generative tasks, the heading component is handled relatively, but the camera’s absolute tilt relative to gravity is still ambiguous. AOE is introduced specifically to remove that ambiguity. It provides each token with two per-token maps: a Latitude map, which measures the elevation angle of the viewing ray above or below the horizontal plane, and an Up map, which measures the direction in image-space corresponding to the world up vector. Together, pitch and roll are described as sufficient to fix the camera’s orientation around the two axes orthogonal to yaw; by anchoring “up” to gravity, the full 3-D rotation is specified (Zhang et al., 8 Dec 2025).

This makes AOE a representation of absolute orientation in the strict sense of gravity-referenced tilt, rather than merely relative inter-frame motion. Within UCPE, that distinction is central: relative camera motion and absolute orientation are separated, then recombined into a single token-level geometry encoding.

2. Construction from world-space rays

AOE is defined per token from the world-space unit ray

$$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$

The first component is the latitude map: $$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$ By construction, $\mathrm{Lat}_t>0$ for rays above the horizon and $\mathrm{Lat}_t<0$ for rays below it.

The second component is the up-vector map. UCPE defines the world-up axis as

$\mathbf{u}^{\mathrm{wld}}=[0,-1,0]^\top.$

For each token, a tangent axis is formed by

$$\mathbf{k}_t=\mathbf{d}_t\times \mathbf{u}^{\mathrm{wld}},$$

which is then normalized. The ray $\mathbf{d}_t$ is rotated by a small angle $\delta$ about that normalized axis using Rodrigues’ formula,

$SE(3)$0

Both $SE(3)$1 and $SE(3)$2 are projected back to the image plane with the same UCM projection $SE(3)$3, yielding $SE(3)$4 and $SE(3)$5. Their displacement is

$SE(3)$6

and the normalized 2-D direction gives

$SE(3)$7

The complete AOE feature for token $SE(3)$8 is therefore

$SE(3)$9

This construction is notable because it is both continuous and per-token. Rather than assigning a single camera-wide pitch-roll label, it derives gravity-anchored orientation cues from the geometry of individual viewing rays. That design is consistent with UCPE’s broader goal of unifying 6-DoF poses, intrinsics, and lens distortions in a geometry-consistent representation (Zhang et al., 8 Dec 2025).

3. Combination with Relative Ray Encoding and transformer injection

UCPE defines the full token encoding as a hybrid of three components: Relative Ray Encoding $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$0, standard image-space RoPE $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$1, and AOE. The ray and RoPE parts are assembled into a block-diagonal operator,

$$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$2

This operator is injected into attention as in GTA: $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$3 AOE itself enters as a per-token bias: $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$4 where $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$5 is a small linear layer projecting $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$6.

The implementation in the pretrained Wan2.1 DiT freezes the original model weights and appends a parallel camera-aware branch to each self-attention block. Input tokens are projected by three small linear layers $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$7 to a reduced dimension $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$8; geometric modulation with $$\mathbf{d}_t = [d_{t,x},\, d_{t,y},\, d_{t,z}]^\top,\qquad \|\mathbf{d}_t\|=1.$$9 and the AOE bias is applied; lightweight attention is computed on the compressed $$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$0; the result is projected back to $$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$1 by a zero-initialized linear layer and added residually to the original self-attention output. Because the output projection is zero-initialized, the pretrained model is unchanged at initialization, and only the adapter parameters are trained. The paper states that this adds less than $$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$2 trainable parameters (Zhang et al., 8 Dec 2025).

Architecturally, this means AOE is neither a substitute for relative geometry nor a standalone positional code. It is a gravity-anchored augmentation inside a broader geometry operator that already encodes full 6-DoF pose, intrinsics, and distortion.

4. Experimental behavior and ablations

On the synthesized dataset reported in Table 1 of the UCPE paper, adding AOE while keeping the adapter size fixed at $$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$3M parameters improves both orientation accuracy and generation fidelity (Zhang et al., 8 Dec 2025).

Metric	UCPE w/o AOE	UCPE w/ AOE
Pitch error	$$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$4	$$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$5
Roll error	$$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$6	$$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$7
FoV error	$$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$8	$$\mathrm{Lat}_t = \arctan2\!\bigl(-d_{t,y},\,\sqrt{d_{t,x}^2+d_{t,z}^2}\bigr).$$9
$\mathrm{Lat}_t>0$0 error	$\mathrm{Lat}_t>0$1	$\mathrm{Lat}_t>0$2
Relative RotErr	$\mathrm{Lat}_t>0$3	$\mathrm{Lat}_t>0$4
Video FVD	$\mathrm{Lat}_t>0$5	$\mathrm{Lat}_t>0$6
FID	$\mathrm{Lat}_t>0$7	$\mathrm{Lat}_t>0$8

The ablations further report that removing the Lat-Up map, or moving the adapter “before” or “after” attention, degrades both orientation accuracy and video quality. Replacing AOE with only PRoPE or GTA likewise loses lens and pose fidelity under non-pinhole projections. These findings delimit AOE’s contribution fairly clearly: it is most impactful on global orientation control, while preserving low relative rotation error and improving video quality simultaneously (Zhang et al., 8 Dec 2025).

A common misreading would be to treat AOE as responsible for all aspects of camera control. The reported results do not support that interpretation. The paper instead presents full control as the outcome of the combination of Relative Ray Encoding with AOE, where AOE specifically resolves the pitch-roll ambiguity and strengthens fidelity under diverse camera models.

5. Relation to other orientation-encoding formulations

The phrase absolute orientation has established meanings in several other technical literatures, but those meanings differ substantially from AOE in UCPE.

In ontology matching, absolute orientation denotes the estimation of a rigid or similarity transform between two point sets $\mathrm{Lat}_t>0$9 and $\mathrm{Lat}_t<0$0, solving

$\mathrm{Lat}_t<0$1

by centering, cross-covariance construction, SVD, optional scale estimation, and translation recovery. That procedure is then used to align ontology embedding spaces before nearest-neighbor matching (Portisch et al., 2022). Here, “absolute orientation” is an alignment algorithm in $\mathrm{Lat}_t<0$2, not a token-wise camera encoding.

In polarized-skylight navigation, absolute orientation is encoded as a cyclical heading variable by an exponential-function neuron bank,

$\mathrm{Lat}_t<0$3

trained with an MSE loss over all heading neurons. The purpose is biologically inspired compass coding of heading, not camera tilt control (Liang et al., 2021).

In computational neuroscience, absolute orientation appears as a property of an orientation complex built from place-cell and head-direction coactivity. Pose simplexes $\mathrm{Lat}_t<0$4 form a simplicial representation converging to a nerve over $\mathrm{Lat}_t<0$5, with persistent and zigzag persistence used to analyze the learning dynamics of orientation structure (Dabaghian, 2021).

In DNA origami self-assembly, absolute orientation means that all degrees of freedom of a nanoscale device are specified by a shape-matched energy landscape with a unique maximum, allowing orientation within $\mathrm{Lat}_t<0$6 on $\mathrm{Lat}_t<0$7 and arbitrary per-device assignment across arrays (Gopinath et al., 2018).

In object-orientation foundation models, absolute orientation is parameterized by discretized azimuth, elevation, and in-plane rotation, with a symmetry-aware periodic target distribution over azimuth bins to represent $\mathrm{Lat}_t<0$8-fold rotational symmetry (Wang et al., 9 Jan 2026).

This suggests that “absolute orientation” is a domain-dependent concept. In UCPE, the expression Absolute Orientation Encoding refers narrowly to a gravity-anchored representation of camera pitch and roll; it is not synonymous with rigid point-set alignment, biological heading codes, discrete cognitive-map orientation, nanoscale placement control, or symmetry-aware object pose distributions.

6. Scope, limitations, and broader implications

Within UCPE, AOE is presented as one component of a broader Unified Camera Positional Encoding. The paper states that Relative Ray Encoding unifies complete camera information, including 6-DoF poses, intrinsics, and lens distortions, while AOE identifies pitch and roll as the two components needed for full control over the initial camera orientation. Together, these components are used for camera-controlled text-to-video generation, and the resulting system achieves state-of-the-art camera controllability and visual fidelity while training only a lightweight adapter (Zhang et al., 8 Dec 2025).

The limits of the method are also explicit in the design. AOE does not replace relative geometry; it is added as a bias on top of the ray-based operator. It does not independently encode lens parameters; those remain in the Relative Ray Encoding and UCM-based projection pipeline. The ablations further indicate that the method is sensitive to where the adapter is inserted and that simplified alternatives such as PRoPE or GTA do not preserve lens and pose fidelity under non-pinhole projections. A plausible implication is that absolute camera orientation cannot be handled robustly as a purely image-space positional signal when diverse intrinsics and distortions are present.

The paper also constructs a large video dataset covering a wide range of camera motions and lens types and highlights UCPE’s potential as a general camera representation for Transformers across future multi-view, video, and 3D tasks (Zhang et al., 8 Dec 2025). In that broader perspective, AOE can be understood as a compact mechanism for injecting gravity-referenced orientation cues into tokenized visual models without retraining the backbone. Its specific contribution is not to generalize all of camera geometry, but to make global tilt explicit where relative pose alone leaves it underdetermined.