InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset

Abstract: Camera extrinsic calibration is a fundamental task in computer vision. However, precise relative pose estimation in constrained, highly distorted environments, such as in-cabin automotive monitoring (ICAM), remains challenging. We present InCaRPose, a Transformer-based architecture designed for robust relative pose prediction between image pairs, which can be used for camera extrinsic calibration. By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder, our model effectively captures the geometric relationship between a reference and a target view. Unlike traditional methods, our approach achieves absolute metric-scale translation within the physically plausible adjustment range of in-cabin camera mounts in a single inference step, which is critical for ICAM, where accurate real-world distances are required for safety-relevant perception. We specifically address the challenges of highly distorted fisheye cameras in automotive interiors by training exclusively on synthetic data. Our model is capable of generalization to real-world cabin environments without relying on the exact same camera intrinsics and additionally achieves competitive performance on the public 7-Scenes dataset. Despite having limited training data, InCaRPose maintains high precision in both rotation and translation, even with a ViT-Small backbone. This enables real-time performance for time-critical inference, such as driver monitoring in supervised autonomous driving. We release our real-world In-Cabin-Pose test dataset consisting of highly distorted vehicle-interior images and our code at https://github.com/felixstillger/InCaRPose.

• The paper introduces InCaRPose, a Transformer-based architecture for metric-scale relative camera pose estimation in automotive cabins.
• It leverages synthetic data and frozen vision transformers with cross-attention to achieve robust real-time calibration under distortion and occlusion.
• Experimental results show low median errors (0.07 m translation, 2.75° rotation) and superior performance compared to classical SIFT and Reloc3r methods.

InCaRPose: Data-Efficient Relative Camera Pose Estimation for Automotive Cabin Environments

Problem Formulation and Context

Interior cabin camera calibration is critical for automotive perception tasks such as driver monitoring, gaze estimation, and occupant-aware safety mechanisms. Extrinsic calibration is particularly challenging due to dynamic camera mounts (e.g., rear-view mirror adjustment), high distortion (fisheye optics), and domain shifts between synthetic and real imagery. Traditional geometric pipelines degrade under occlusion and distortion, while deep neural architectures often depend heavily on large training corpora and fixed camera intrinsics.

The paper introduces InCaRPose—an end-to-end Transformer-based architecture specifically designed for robust, metric-scale relative camera pose prediction in complex vehicle interiors. InCaRPose addresses automotive calibration by reformulating pose prediction as reference-relative estimation, thereby bypassing vehicle-specific coordinate frames and enabling generalization across platforms.

Figure 1: InCaRPose predicts the relative camera pose between a reference and target view using synthetic training, generalizing to real cabin environments without retraining.

Figure 2: Visualization of the camera and vehicle coordinate systems, highlighting the need for vehicle-agnostic calibration.

Dataset Construction and Synthetic-to-Real Transfer

The InCaRPose model is trained exclusively on synthetic data rendered from diverse automobile cabin scenes in Blender, capturing a wide range of rotation ($\pm80^\circ$ x/y, $\pm50^\circ$ z) and translation ($\pm20$ cm per axis) relative to a canonical cabin view. The test set comprises real-world cabin sequences, systematically covering 6-DoF transformations, acquired under stringent conditions with ground truth via ArUco markers for metric-scale reliability.

Figure 3: Comparison of real-world and synthetic cabin views, demonstrating domain similarity and transfer potential.

Key advantages of the released dataset—In-Cabin-Pose—include high distortion, wide-FoV NIR fisheye imagery, and rigorous metric ground truth. COLMAP-based Structure-from-Motion is used for contrast but shown to be unreliable in constrained interiors due to scale ambiguity and drift, particularly along the vertical axis.

Model Architecture

InCaRPose utilizes a frozen, pre-trained vision transformer backbone (DINOv3, Dune, DUST3R) to extract robust patch-wise features. A cross-attention Transformer decoder fuses these features, modeling spatial relationships between image pairs. The lightweight MLP prediction head regresses a pose vector, with support for multiple output formats—rotation vector, Euler angles, quaternions, and rotation matrices—followed by parameter-specific post-processing (e.g., quaternion normalization, SVD orthogonalization).

Figure 4: Architecture overview. Two images pass through a frozen ViT backbone and cross-attention decoder, finishing with a prediction head that outputs the relative pose.

Information is distilled via a convolutional bottleneck and global pooling. Bidirectional pose output enforces geometric consistency. Only the decoder and head are trained, ensuring backbone feature generalization for synthetic-to-real transfer.

Experimental Evaluation

The model achieves real-time inference ($\geq$67 FPS on RTX4090 at 224 resolution) with minimal latency (~15 ms per frame). Ablation studies on backbone architecture (ViT-Small/Base/Large, Dune-Base, DUST3R-Large), rotational parameterization, and image resolution are provided.

Figure 5: Qualitative results on real-world inference. Translation vectors are scale-normalized for coordinate frame comparison.

On the In-Cabin-Pose dataset, InCaRPoseLarge224 attains a median translation error of $0.07$ m and rotation error of $2.75^\circ$, outperforming classical SIFT matching—which is scale ambiguous—and Reloc3r, which predicts only translation direction (not metric scale) and benefits mainly from undistorted input.

Figure 6: COLMAP fails to recover vertical translation in cabin scenes, highlighting the need for robust metric estimation.

The model maintains performance on images without visible markers, confirming agnosticism to marker cues and robustness to distractors. Backbone scaling improves performance chiefly in low-data regimes; in structured environments (e.g., 7-Scenes) the gains from larger backbones and higher input resolution are less pronounced.

Figure 7: Rotation error versus parameter count, illustrating accuracy-latency trade-offs.

Benchmarking on Public Datasets

On public indoor datasets (7-Scenes), InCaRPoseLarge224 achieves median rotation error of $2.25^\circ$ and translation error of $0.13$ m, significantly outperforming Reloc3r (median $2.45^\circ$ rotation, direction only). Outdoor Cambridge Landmarks evaluation reveals higher errors due to larger viewpoint gaps and reduced feature recurrence, e.g., mean rotation error over $6^\circ$ and translation error over $\pm50^\circ$0 m. InCaRPose matches or surpasses prior methods on indoor settings but remains sensitive to challenging scene transitions.

Representation and Backbone Ablations

Ablation studies reveal that quaternion output with dedicated loss yields lowest rotation error ($\pm50^\circ$1 median). Dune-Base backbone further improves performance over DINOv3. Increasing input resolution improves rotational accuracy but not translation error, suggesting a bottleneck in metric inference due to environmental constraints rather than pixel density.

Practical and Theoretical Implications

From a practical standpoint, InCaRPose enables real-time, metric-scale calibration for dynamic in-cabin environments without undistortion, retraining, or large-scale data. Safety-critical applications—like occupant-aware airbag deployment—benefit from robust pose estimates within strict latency windows. The model’s synthetic-to-real transfer capacity minimizes the need for invasive real-world data collection, facilitating rapid prototyping across vehicle platforms.

Theoretically, the success of frozen self-supervised backbone features and cross-attention mechanisms underscores the representational power of foundation models for metric estimation tasks with limited domain data. The reference-relative formulation and bidirectional pose outputs enable vehicle-agnostic deployment and facilitate generalization.

Figure 8: Example of center-crop distortion, highlighting the importance of preserving fisheye corner cues.

Future Directions

Robustness under partial occlusion and dynamic cabin conditions remains an open challenge. Further development will focus on domain-specific occlusion test sets and architectural enhancements for stronger spatial reasoning under extreme viewpoint shifts and crowded environments. Transfer to outdoor settings and highly variable appearances (e.g., Shop Facade, Cambridge Landmarks) will require advances in foundation model capabilities and larger synthetic scene generation.

Figure 9: Visualization of x-axis translation in ground truth versus COLMAP, showing dependency on feature density.

Conclusion

InCaRPose delivers efficient, real-time relative pose estimation for in-cabin automotive applications, operating directly on highly distorted fisheye images. Leveraging frozen vision transformers and cross-attention, it achieves competitive accuracy with minimal data, robust synthetic-to-real transfer, and vehicle-agnostic deployment. Limitations in outdoor and occluded environments point towards future toolbox expansion. The work advances the methodological state-of-the-art for reliable, scalable camera calibration in constrained and dynamic contexts.