3DRot: Geometry-Consistent 3D Augmentation

• 3DRot is a plug-and-play data augmentation method that applies camera-centric 3D rotations and reflections to maintain geometric consistency in RGB images and 3D annotations.
• It synchronously updates the RGB image, camera intrinsics, and object poses, ensuring that all parameters remain aligned for accurate 3D task performance.
• Empirical results on SUN RGB-D demonstrate improved IoU₍₃D₎, reduced rotation errors, and higher mAP₍0.5₎, equating to gains similar to using extra training data.

3DRot is a plug-and-play data augmentation method for RGB-based 3D tasks that applies camera-centric 3D rotations and reflections while preserving projective geometry by synchronously updating the RGB image, camera intrinsics, object poses, and 3D annotations. It was introduced to address a central limitation of RGB-based 3D perception: annotations are scarce and expensive, while the available augmentation toolbox is thin because common image transforms, including resize and rotation, disrupt geometric consistency. The method is formulated around rotations and mirroring about the camera’s optical center, achieves geometry-consistent rotations and reflections without relying on any scene depth, and was validated on monocular 3D detection on SUN RGB-D, where it improved $IoU_{3D}$, reduced rotation error, and increased $mAP_{0.5}$ (Yang et al., 2 Aug 2025).

1. Problem formulation and motivation

RGB-based 3D tasks such as 3D detection, depth estimation, and 3D keypoint estimation require consistency between the image plane, camera parameters, and 3D annotations. The motivation for 3DRot is that most standard 2D augmentations alter that relationship unless all associated parameters are synchronously updated. In the formulation reported for 3DRot, naive image rotation, resize, crop, or flipping can destroy projective consistency because a transformed 2D box need not correspond to a physically valid 3D box in the camera coordinate frame (Yang et al., 2 Aug 2025).

The method therefore treats augmentation as a camera-space transformation rather than an image-only perturbation. Its central claim is that images can be rotated and mirrored about the camera’s optical center while keeping all 3D-to-2D correspondences valid, provided that the RGB image, camera intrinsics, object poses, and other 3D annotations are transformed together. This positioning distinguishes 3DRot from ordinary image augmentation pipelines, which typically modify only pixels and 2D labels.

A further motivation is the avoidance of depth dependence. The method explicitly targets geometry-consistent augmentation without requiring per-pixel depth or full scene reconstruction. That design choice is important for RGB-only settings, where depth is often unavailable or unreliable.

2. Camera-centric 3D transformation model

3DRot applies 3D rotations in pitch, yaw, and roll, as well as mirroring, in camera space. Let a world point be denoted by $T_w$, with original camera-space coordinates

$T_c = R\,T_w + t.$

Under a camera-centric augmentation rotation $R_c$, the transformed point becomes

$T_{rot-c} = R_c\,T_c = R_c R\,T_w + R_c t.$

This updates the effective camera extrinsics and, correspondingly, the relative pose of every object in the scene (Yang et al., 2 Aug 2025).

For a 3D bounding box parameterized as

$T_{bb} = R_{bb} S_{bb} + t_{bb},$

the rotated box is

$$T_{bb}^{new} = R_c(R_{bb} S_{bb} + t_{bb}) = (R_c R_{bb}) S_{bb} + R_c t_{bb}.$$

Accordingly,

$R_{new} = R_c R_{bb}, \qquad t_{new} = R_c t_{bb},$

with side lengths unchanged. In the reported formulation, this is the key mechanism by which object orientation and translation are updated consistently under augmentation.

The image warp is derived from a rotation-induced homography. The general planar homography is written as

$$P_A = \frac{z_B}{z_A} K_A H_{AB} K_B^{-1} P_B, \qquad H_{AB} = R_{AB} - \frac{t_{AB} n^T}{d}.$$

For pure camera rotation, $mAP_{0.5}$0, so $mAP_{0.5}$1. The resulting image mapping is therefore

$mAP_{0.5}$2

where $mAP_{0.5}$3 is the homogeneous normalization scalar. The significance of this formulation is that the warp does not rely on scene planarity or depth when the augmentation is a pure camera rotation (Yang et al., 2 Aug 2025).

3. Synchronous update of images, intrinsics, and 3D annotations

The operational definition of 3DRot is a synchronous transformation of all modalities associated with a training sample. The RGB image is warped using the homography induced by the camera-space transform. Camera intrinsics are updated when the image footprint changes after rotation or reflection. Object poses are rotated accordingly. Other 3D annotations, including keypoints or related camera-space labels, are transformed by the same rotation or reflection (Yang et al., 2 Aug 2025).

This synchronization is the method’s primary technical constraint and its main contribution. In effect, augmentation is applied to the camera and scene relationship rather than to the image alone. Because all labels and calibration parameters are moved together, the transformed sample remains valid for 3D supervision.

The paper’s exposition also stresses that the method is not tied to a specific label format. A plausible implication is that any representation expressible in camera coordinates can be transformed by the same camera-space operator, provided the underlying training pipeline consumes synchronized intrinsics and 3D annotations.

4. Reflection, chirality, and principal-point realignment

3DRot treats reflection separately from rotation because mirroring changes chirality. The reflection transform is expressed with a mirror matrix $mAP_{0.5}$4, so that the transformed point is

$mAP_{0.5}$5

An object pose $mAP_{0.5}$6 is updated as

$mAP_{0.5}$7

and under combined rotation and reflection as

$mAP_{0.5}$8

The corresponding world-to-image mapping is written as

$mAP_{0.5}$9

The explicit treatment of chirality is necessary because a conventional horizontal flip in image space does not by itself produce a correct 3D pose update (Yang et al., 2 Aug 2025).

The method also addresses a practical consequence of out-of-plane rotation: pitch and roll can move valid content outside the original image bounds or shift it away from the principal point. To compensate, 3DRot computes the warped image’s exact minimal bounding box, pads or crops as needed, and updates the camera intrinsics to keep the principal point at the image center. The details further report that image borders are projected through the new intrinsics and then realigned so that valid pixels are preserved.

This step is not merely cosmetic. It preserves consistency between the warped image footprint and the intrinsics supplied to the downstream model. The reported ablations indicate that principal-point realignment provides an additional small boost, while both rotation and chirality-correct flipping contribute to the final performance.

5. Empirical evaluation on monocular 3D detection

3DRot was validated on monocular 3D detection on the SUN RGB-D dataset. The reported results compare a SUN10 baseline with the same system augmented by 3DRot, and also include a comparison drawn against Cube R-CNN results reported with larger training data (Yang et al., 2 Aug 2025).

Method	$T_w$0	ROT	$T_w$1
SUN10 Baseline	43.21	22.91°	35.70
SUN10 + 3DRot	44.51	20.93°	38.11
Cube R-CNN (SUN RGB-D)	36.2	-	34.7
Cube R-CNN (multi-data)	37.8	-	35.4

On SUN RGB-D, 3DRot raises $T_w$2 from 43.21 to 44.51, cuts rotation error from $T_w$3 to $T_w$4, and boosts $T_w$5 from 35.70 to 38.11. The details characterize these as gains of +1.3 $T_w$6, approximately $T_w$7 ROT, and +2.4 absolute $T_w$8. The same source also reports that Cube R-CNN, after adding three other datasets together with SUN RGB-D, increases $T_w$9 from 36.2 to 37.8 and $T_c = R\,T_w + t.$0 from 34.7 to 35.4. Within the reported comparison, this frames 3DRot as producing gains on par with, or greater than, the effect of adding substantial extra training data.

The evaluation is limited to a classical RGB-based 3D task, namely monocular 3D detection. Nevertheless, because the method is defined through camera-space transforms rather than architecture-specific modifications, the authors state that it is readily transferable to other 3D tasks.

6. Scope, misconceptions, and relation to adjacent rotation-aware research

A common misconception is to view 3DRot as ordinary image rotation. In the method’s own formulation, that is incorrect: the transformation is valid because the RGB image, camera intrinsics, object poses, and 3D annotations are all updated synchronously. Another misconception is that geometry-consistent augmentation of this kind must require depth. The reported homography construction under pure camera rotation is presented precisely to avoid that dependency (Yang et al., 2 Aug 2025).

A second misconception is that rotation handling in 3D perception is synonymous with detector-level rotation invariance. Related work illustrates a broader design space. In LiDAR-based 3D object detection, RIDE introduces a bi-feature extractor combining object-aware features with rotation-invariant features, and reports improvements of +5.6% mAP and 53% rotation robustness on KITTI, as well as +5.1% and 28% on nuScenes (Wang et al., 2024). In aerial manipulation, Rotation-SqueezeDet extends the detection tuple to $T_c = R\,T_w + t.$1 and reports 78.0% AP at IoU $T_c = R\,T_w + t.$2 on the UAV-BD dataset while enabling rotation-aware grasping (Lin et al., 2018). This suggests a complementary taxonomy: 3DRot operates at the data level through camera-space augmentation, whereas other methods address rotation through invariant features or explicit orientation prediction.

The method should also be distinguished terminologically from 3DRO, which denotes an SE(3) radar odometry framework using a 2D imaging radar and a gyroscope rather than an augmentation method (Gentil et al., 13 Apr 2026). The similarity of acronyms can obscure substantial differences in problem setting, sensing modality, and mathematical objective.

Within RGB-based 3D learning, 3DRot’s specific contribution is therefore not rotation estimation as such, nor rotation-invariant feature extraction, but a mathematically synchronized augmentation mechanism that preserves projective geometry while expanding the usable augmentation space.