---
title: 'InCaRPose: In-Cabin Pose Estimation'
url: https://www.emergentmind.com/topics/incarpose
type: topic
---

# InCaRPose: In-Cabin Pose Estimation

Searching arXiv for the specified paper to ground the article and citation.
InCaRPose is a learning-based framework for estimating the relative 6-DoF pose between two in-cabin camera views, introduced as a method for camera extrinsic calibration in automobile interiors [2604.03814]. It is designed for settings in which wide-angle or fisheye NIR cameras mounted on a movable rear-view mirror undergo repeated, safety-relevant pose changes, making robust recovery of both rotation and metric translation necessary for downstream perception tasks such as driver monitoring and occupant positioning. The method combines a frozen self-supervised ViT backbone with a Transformer-based decoder and a quaternion-based metric pose objective, and it is trained exclusively on synthetic data while targeting generalization to real-world cabin environments [2604.03814].

## 1. Problem setting and calibration objective

Camera extrinsic calibration is the underlying task addressed by InCaRPose. In the in-cabin automotive monitoring setting, the challenge is not merely relative orientation recovery, but precise estimation of the full relative transform between a calibrated “standard” view and a shifted view acquired after a mount displacement. The target quantity is a relative rigid transform $T_{\mathrm{rel}} \in SE(3)$ satisfying
$$
T_{v2} = T_{v1} \cdot T_{\mathrm{rel}},
$$
where $T_{v1}$ is a known calibrated reference pose and $T_{v2}$ is the shifted pose [2604.03814].

The motivating conditions are specific to ICAM. In-cabin cameras often use fisheye lenses with up to $180^\circ$ field of view, and the scene contains strong distortion, constrained baselines, and frequent occlusions from visors, hands, and headrests. According to the reported formulation, these factors degrade classical pipelines based on undistortion, feature matching, and essential-matrix decomposition, while incremental SfM cannot recover metric scale without a 3D map [2604.03814].

The operational constraints are also safety-critical. Mounting variability caused by a camera attached to a movable mirror leads to unpredictable on-the-fly shifts of the extrinsic pose. For applications such as gaze estimation and airbag actuation, the relevant requirement is metric accuracy at real-world scale together with real-time inference, specifically $\leq 15\,\mathrm{ms}$ for inference within a $15$–$50\,\mathrm{ms}$ end-to-end response window in crash scenarios [2604.03814]. Within that context, a single-step regressor that directly predicts metric relative pose is presented as an alternative to iterative optimization, scene reconstruction, or explicit undistortion.

## 2. Architecture and computational design

InCaRPose is organized into three modules: a frozen DINOv3 ViT backbone, a Transformer-based decoder, and a prediction head [2604.03814]. The backbone, instantiated with ViT-Small, Base, or Large variants, encodes the reference and target images into patch-token sets $X_{\mathrm{ref}}, X_{\mathrm{tgt}} \in \mathbb{R}^{N \times D}$. The backbone remains frozen, and the stated rationale is to preserve strong self-supervised geometric features while enabling data-efficient training on only approximately $6.5\mathrm{k}$ synthetic image pairs [2604.03814].

The decoder consists of $L$ stacked decoder blocks. Each block includes multi-head self-attention over the concatenated token set, cross-attention in both directions between reference and target tokens, 2D rotary positional embeddings on queries and keys, and standard residual connections with LayerNorm [2604.03814]. This design is intended to capture relative geometric cues directly from paired views rather than from hand-crafted correspondences.

After the decoder, the representation passes through a residual convolutional bottleneck for dimensionality reduction, followed by global average pooling, LayerNorm, GELU, and a linear layer that produces a pose vector $y \in \mathbb{R}^d$ [2604.03814]. During training, the model can optionally predict both $T_{\mathrm{rel}}$ and its inverse in one forward pass to enforce geometric consistency. The output is described as “quaternion + translation,” indicating direct regression of orientation and metric displacement in a single inference step.

A plausible implication is that the architecture is positioned between generic image-pair regression and classical geometric estimation: it retains explicit pairwise structure through cross-attention while avoiding correspondence extraction and scale ambiguity.

## 3. Pose parameterization and optimization criterion

The pose representation used in InCaRPose consists of a unit quaternion $q \in S^3$ for rotation and a translation vector $t \in \mathbb{R}^3$ in metric units, assembled as
$$
y = [q, t].
$$
Post-processing enforces $\|q\| = 1$ [2604.03814].

Ground-truth relative pose is derived from the absolute poses $T_{v1}$ and $T_{v2}$ through
$$
T_{\mathrm{rel}} =
\begin{pmatrix}
R_{\mathrm{rel}} & t_{\mathrm{rel}} \\
0 & 1
\end{pmatrix},
$$
with
$$
R_{\mathrm{rel}} = R_{v1}^{\top} R_{v2}, \qquad
t_{\mathrm{rel}} = R_{v1}^{\top}(t_{v2} - t_{v1}).
$$
This formulation expresses translation in the coordinate system of the reference camera and directly supervises the target relative motion [2604.03814].

The loss is a quaternion-based pose loss,
$$
L_{\mathrm{quat}} = E[e_{\mathrm{rot\_quat}}] + \alpha \cdot E[e_{\mathrm{trans\_eucl}}],
$$
where
$$
e_{\mathrm{rot\_quat}}(\hat q, q) = 2 \cdot \arccos(|\hat q^{\top} q|),
\qquad
e_{\mathrm{trans\_eucl}}(\hat t, t) = \|\hat t - t\|_2,
$$
and $\alpha$ is set empirically to $1$ [2604.03814]. The reported interpretation is that this objective yields stable gradients on $S^3$ and directly supervises metric translation.

An ablation reported in the same source compares rotation vectors, Euler angles, quaternions, and 9-DoF matrix outputs. The quaternion representation together with the quaternion loss gives the lowest median rotation error, specifically $3.55^\circ$ versus $4.67$–$7.61^\circ$ for the alternatives [2604.03814]. Within the paper’s scope, that result supports the selected parameterization as the preferred representation for relative-pose regression under the stated constraints.

## 4. Synthetic data regime and real-world generalization

Training is performed exclusively on synthetic data. The synthetic cabin dataset comprises eight distinct car interiors modeled in Blender. For each standard view, the generation process uniformly samples rotations of $\pm 80^\circ$ about $X,Y$ and $\pm 50^\circ$ about $Z$, together with translations of $\pm 20\,\mathrm{cm}$ along each camera axis, yielding approximately $5\mathrm{k}$ rotation-only and $1.5\mathrm{k}$ rotation-plus-translation pairs for training, plus approximately $2\mathrm{k}$ validation pairs [2604.03814].

The rendering process randomizes cabin occupants, objects, textures, and NIR lighting to cover occlusions and varied reflectance. Fisheye intrinsics and distortion parameters are randomly sampled per render, images are rendered with full border distortion, and zero-padding rather than center-cropping is used to preserve extreme corners. ColorJitter over brightness, contrast, saturation, and hue is applied during training [2604.03814].

No real images or precise intrinsics are used during training. Nevertheless, the framework is reported to generalize to real NIR fisheye interiors without undistortion or camera-specific fine-tuning [2604.03814]. The stated explanation is that the frozen DINOv3 backbone provides strong domain-invariant features and that aggressive domain randomization bridges the synthetic-to-real gap.

This suggests that InCaRPose should be understood not only as a pose regressor but also as an instance of synthetic-data-driven calibration, in which robustness to distortion and intrinsics mismatch is treated as a representation-learning problem rather than as a pre-calibration prerequisite.

## 5. Empirical performance across datasets

The principal evaluation is conducted on the real-world In-Cabin-Pose test set, which contains $550$ images with ARUCO marker ground truth [2604.03814]. The reported median errors are summarized below.

| Variant | Rotation error | Translation error |
|---|---:|---:|
| InCaRPoseLarge224 | $2.75^\circ$ | $0.07\,\mathrm{m}$ |
| InCaRPoseBase224 | $3.55^\circ$ | $0.09\,\mathrm{m}$ |
| InCaRPoseSmall224 | $4.43^\circ$ | $0.08\,\mathrm{m}$ |

For the same in-cabin benchmark, InCaRPoseLarge224 also reports a median translation direction error of $23.5^\circ$ [2604.03814]. The classical SIFT plus essential-matrix baseline recovers rotation with approximately $4.8^\circ$ median error but does not recover metric translation. Reloc3r, described as a large-data baseline, also cannot recover scale and requires undistortion to approach $11.6^\circ$ median direction error [2604.03814]. The comparison therefore distinguishes between orientation recovery and metric-scale pose estimation, with InCaRPose explicitly targeting the latter.

On the public 7-Scenes benchmark, InCaRPoseLarge512 achieves mean rotation error $2.21^\circ$, median translation error $0.15\,\mathrm{m}$, and median direction error $10.35^\circ$ [2604.03814]. The paper states a $-65\%$ rotation improvement relative to Reloc3r512’s $6.37^\circ$, and further states that all InCaRPose variants outperform Reloc3r224/512 on rotation while recovering metric translation [2604.03814]. This places the method outside a purely in-cabin niche and indicates competitive behavior on a public indoor relocalization benchmark.

On Cambridge Landmarks, the reported aggregated performance is approximately $6.5^\circ$ mean rotation error for InCaRPoseLarge224 and approximately $1.2\,\mathrm{m}$ median translation error [2604.03814]. Performance degrades outdoors, which the source attributes to sparser features and larger scale variation, although the method remains competitive on ShopFacade. A plausible implication is that the model’s inductive biases and training regime are particularly aligned with dense, geometrically constrained interiors rather than broad outdoor viewpoint changes.

## 6. Ablations, runtime, and release artifacts

Several ablation results clarify the model’s design trade-offs. Larger DINOv3 backbones, progressing from Small to Base to Large, steadily reduce rotation error on the in-cabin data; on large datasets such as 7-Scenes, the gains are smaller [2604.03814]. The multi-teacher DUNE-Base encoder slightly outperforms DINOv3-Base, and increasing input resolution from $224$ to $512$ pixels improves rotational accuracy while having only marginal effect on translation [2604.03814]. These results indicate that orientation estimation benefits more directly from representational scale and spatial detail than metric translation does under the tested conditions.

Training uses batch size $8$, the AdamW optimizer with learning rate $1 \times 10^{-6}$, weight decay $1 \times 10^{-5}$, and $50\%$ dropout in the MLP head; only the decoder and head are trained, while the ViT backbone remains frozen [2604.03814]. This training configuration is consistent with the data-efficient design choice described earlier.

Inference speed is reported on an NVIDIA RTX 4090. InCaRPoseSmall224 runs at approximately $68$ FPS ($14.6\,\mathrm{ms}$/frame), Base224 at approximately $67$ FPS, and Large224 at approximately $46$ FPS. With FP16 and `torch.compile`, the Large variant can reach approximately $68$ FPS [2604.03814]. The source concludes that all variants exceed real-time requirements for in-cabin streams.

The release artifacts include the In-Cabin-Pose test dataset and the codebase. The dataset contains $550$ high-distortion NIR fisheye image pairs with ARUCO and COLMAP ground truth, systematically covering $\pm 20\,\mathrm{cm}$ translations and $\pm 80^\circ$ rotations along each axis, and the code and synthetic data generation scripts are made available through the project repository [2604.03814]. The inclusion of both real-world evaluation data and synthetic-data tooling is significant because it supports replication of the central claim: a single-step, metric-scale relative-pose predictor trained purely on synthetic fisheye imagery.

## 7. Position within relative-pose estimation research

InCaRPose is presented as a response to several limitations of existing approaches in constrained, highly distorted environments. Classical feature-based methods are described as losing robustness under heavy lens distortion and constrained baselines, while incremental SfM is described as incapable of recovering metric scale without a 3D map [2604.03814]. In that sense, the method addresses not just relative pose in the abstract, but a specific conjunction of requirements: distorted fisheye input, real-time execution, absence of a prior 3D scene model, and direct recovery of metric translation.

Its defining characteristic is the combination of frozen self-supervised visual features, a lightweight Transformer decoder, and a quaternion-based metric loss to estimate 6-DoF relative pose directly from distorted image pairs [2604.03814]. The method is also unusual in being trained on purely synthetic renders yet evaluated on real NIR cabin data without undistortion or camera-specific fine-tuning. This suggests a broader methodological connection to synthetic-to-real transfer in geometric vision, particularly where camera models and distortion profiles vary across deployment settings.

A common misconception in this area is that rotation recovery alone is sufficient for calibration. The reported comparisons explicitly distinguish between estimating orientation and recovering metric translation: SIFT plus essential matrix is said to recover rotation but not metric translation, and Reloc3r is said not to recover scale [2604.03814]. InCaRPose is therefore notable less for replacing all geometric methods universally than for targeting the subset of calibration problems where absolute translation within a physically plausible adjustment range must be inferred in one forward pass.

Within that scope, the method is positioned as an efficient single-step extrinsic calibration solution for safety-critical automotive monitoring, while the reported degradation on outdoor Cambridge Landmarks indicates that its empirical strengths are most evident in dense, bounded interiors rather than unconstrained large-scale scenes [2604.03814].

Source: https://www.emergentmind.com/topics/incarpose