RCTDistill: Cross-Modal 3D Detection Distillation

• RCTDistill is a knowledge distillation framework that leverages LiDAR-based supervision via RAKD, TKD, and RDKD modules to refine spatial, temporal, and relational BEV features.
• The framework significantly boosts 3D detection performance by addressing dynamic object misalignments and employing anisotropic radar-camera uncertainty modeling.
• An alternate route-constrained usage views distillation as curriculum learning over teacher checkpoints, highlighting a distinct staged supervision strategy.

RCTDistill denotes a knowledge distillation framework name that is used most specifically for a cross-modal radar-camera 3D object detection method with temporal fusion, in which a LiDAR-based teacher supervises a radar-camera student through Range-Azimuth Knowledge Distillation (RAKD), Temporal Knowledge Distillation (TKD), and Region-Decoupled Knowledge Distillation (RDKD) (Bang et al., 22 Sep 2025). In related distillation literature, the same label has also been used for a route-constrained distillation interpretation of Route Constrained Optimization, where a student is trained against anchor checkpoints along the teacher’s optimization route rather than only the final converged teacher (Jin et al., 2019). The term therefore spans distinct methodological lineages, with the 2025 usage being a concrete named framework and the earlier usage describing a curriculum-style distillation principle.

1. Scope and nomenclature

The most explicit use of the name appears in "RCTDistill: Cross-Modal Knowledge Distillation Framework for Radar-Camera 3D Object Detection with Temporal Fusion" (Bang et al., 22 Sep 2025). There, RCTDistill is a radar-camera fusion method for 3D object detection that distills LiDAR BEV features into a radar-camera student, and it is organized around three modules: RAKD, TKD, and RDKD.

A separate usage arises from "Knowledge Distillation via Route Constrained Optimization" (Jin et al., 2019). In that work, the method itself is named Route Constrained Optimization, but secondary descriptions identify an RCTDistill interpretation in which distillation is treated as curriculum learning over the teacher’s training trajectory. The student follows selected anchor points on the teacher route in parameter space rather than directly matching only the final teacher state.

This naming overlap matters because the two frameworks solve different problems. The 2025 RCTDistill addresses radar-camera 3D detection under LiDAR teacher supervision and temporal fusion (Bang et al., 22 Sep 2025), whereas the route-constrained formulation addresses generic teacher-student distillation and hint or mimicking learning through staged supervision over teacher checkpoints (Jin et al., 2019). This suggests that "RCTDistill" is not a single standardized term across the distillation literature.

2. Cross-modal RCTDistill for radar-camera 3D detection

In the cross-modal formulation, the teacher is a LiDAR-based CenterPoint detector and the student is a temporal radar-camera BEVFusion-R variant (Bang et al., 22 Sep 2025). The teacher produces low-level and high-level BEV features, denoted $B^{L}_{\text{low}}$ and $B^{L}_{\text{high}}$, from voxelized LiDAR point clouds through a 3D sparse convolution backbone and a 2D CNN decoder. The student combines camera BEV features and radar BEV features through an adaptive gating network, then applies streaming temporal fusion with a memory bank and a Historical Alignment Network (HA-Net), producing low-level fused BEV features $B^{\text{RC}}_{\text{low}}$, temporally aligned features $\hat{B}^{\text{RC}}_{\text{low}}$, and high-level BEV features $B^{\text{RC}}_{\text{high}}$.

The temporal setup uses preceding 8 frames plus the current frame. Historical BEV features are transformed to the current ego frame, concatenated with current low-level BEV features, and aggregated by HA-Net. This is intended to reduce frame-to-frame variation for dynamic objects. At inference time, the teacher is removed and the student runs with radar and camera only.

Module	Supervised representation	Stated purpose
RAKD	$B^{L}_{\text{low}}$ and $B^{\text{RC}}_{\text{low}}$	Refine inaccurate BEV representations using range-azimuth-aware spatial distillation
TKD	$B^{L}_{\text{low}}$ and $\hat{B}^{\text{RC}}_{\text{low}}$	Mitigate temporal misalignment caused by dynamic objects
RDKD	$B^{L}_{\text{high}}$ and $B^{L}_{\text{high}}$0	Distill relational knowledge so the student differentiates foreground and background

The total training loss combines the detection objective with all three distillation terms: $B^{L}_{\text{high}}$1 Here $B^{L}_{\text{high}}$2 is the standard detection loss for BEVFusion-R, and the three auxiliary terms correspond to spatial, temporal, and relational distillation (Bang et al., 22 Sep 2025).

3. Distillation mechanics: range-azimuth, temporal, and region-decoupled supervision

RAKD is designed around anisotropic radar-camera uncertainty. The method models the spatial KD region with ellipse-shaped Gaussian masks in BEV rather than isotropic circular masks. For a ground-truth box with center $B^{L}_{\text{high}}$3, heading $B^{L}_{\text{high}}$4, and dimensions $B^{L}_{\text{high}}$5, the ellipse radii are

$B^{L}_{\text{high}}$6

and the corresponding elliptical Gaussian is

$B^{L}_{\text{high}}$7

with rotated coordinates determined by $B^{L}_{\text{high}}$8 (Bang et al., 22 Sep 2025). After thresholding the merged mask, the low-level student BEV is channel-aligned with a $B^{L}_{\text{high}}$9 convolution and supervised by

$B^{\text{RC}}_{\text{low}}$0

TKD extends this idea along motion trajectories. For dynamic objects, the ellipse center is shifted backward along the velocity vector,

$B^{\text{RC}}_{\text{low}}$1

and the major-axis radius is enlarged to cover the trajectory segment: $B^{\text{RC}}_{\text{low}}$2 This yields a temporal ellipse mask $B^{\text{RC}}_{\text{low}}$3 that supervises the temporally fused student feature $B^{\text{RC}}_{\text{low}}$4 against the current LiDAR feature $B^{\text{RC}}_{\text{low}}$5 through

$B^{\text{RC}}_{\text{low}}$6

RDKD acts on high-level BEV features and distills relational structure rather than direct feature values. The student detection head produces class-wise score maps, which are reduced by a max over classes to $B^{\text{RC}}_{\text{low}}$7. The method selects $B^{\text{RC}}_{\text{low}}$8 locations whose confidence exceeds a threshold $B^{\text{RC}}_{\text{low}}$9, extracts the corresponding feature vectors from teacher and student, and forms cosine-similarity affinity matrices

$\hat{B}^{\text{RC}}_{\text{low}}$0

The relational loss is then

$\hat{B}^{\text{RC}}_{\text{low}}$1

This foreground-focused relational supervision is intended to transfer how LiDAR features separate foreground objects from background clutter (Bang et al., 22 Sep 2025).

4. Empirical performance of the 2025 framework

The 2025 RCTDistill reports state-of-the-art radar-camera fusion performance on both nuScenes and View-of-Delft, with the fastest inference speed of 26.2 FPS (Bang et al., 22 Sep 2025). On nuScenes validation with ResNet-50, image size $\hat{B}^{\text{RC}}_{\text{low}}$2, radar-camera input, and 8+1 frames, the baseline temporal BEVFusion-R achieves NDS 57.3, mAP 50.5, and FPS 27.8, while RCTDistill achieves NDS 62.2, mAP 55.2, and FPS 26.2. On the same split, it exceeds SpaRC in NDS and mAP while retaining higher FPS.

On nuScenes validation with ResNet-101 and image size $\hat{B}^{\text{RC}}_{\text{low}}$3, the baseline achieves NDS 60.9 and mAP 54.2, SpaRC reaches NDS 64.4 and mAP 57.1, and RCTDistill reaches NDS 65.5 and mAP 59.0 at 8.4 FPS (Bang et al., 22 Sep 2025). A single-frame variant, RCTDistill-S, also outperforms prior KD methods: with ResNet-50 and $\hat{B}^{\text{RC}}_{\text{low}}$4, it reports NDS 55.5 and mAP 46.4, exceeding X3KD and CRKD while running at 28.0 FPS.

On the nuScenes test set with ConvNeXt-B and 8+1 frames, RCTDistill reports NDS 67.4, mAP 60.3, mATE 0.334, mASE 0.245, mAOE 0.334, mAVE 0.251, mAAE 0.113, and FPS 5.0 (Bang et al., 22 Sep 2025). On View-of-Delft it reports EAA AP 62.37 and RoI AP 82.25, surpassing BEVFusion, LXL, HGSFusion, and SGDet3D.

Component ablations show that all three modules contribute. Starting from a baseline of NDS 55.0 and mAP 47.1, RAKD alone yields NDS 57.2 and mAP 49.3, RDKD alone yields NDS 56.8 and mAP 49.0, TKD alone yields NDS 57.0 and mAP 48.5, and the full combination yields NDS 58.4 and mAP 51.0 (Bang et al., 22 Sep 2025). Additional comparisons show that the elliptical Gaussian mask of RAKD outperforms BEVDistill’s circular Gaussian and CRKD MSFD, TKD outperforms VCD and STXD, and RDKD outperforms MonoDistill and whole-map affinity KD.

Setting	Comparator	RCTDistill
nuScenes val, ResNet-50, $\hat{B}^{\text{RC}}_{\text{low}}$5, 8+1	Baseline: NDS 57.3, mAP 50.5, FPS 27.8	NDS 62.2, mAP 55.2, FPS 26.2
nuScenes val, ResNet-101, $\hat{B}^{\text{RC}}_{\text{low}}$6, 8+1	SpaRC: NDS 64.4, mAP 57.1, FPS 7.2	NDS 65.5, mAP 59.0, FPS 8.4
nuScenes test, ConvNeXt-B, 8+1	SpaRC: NDS 67.1, mAP 60.0	NDS 67.4, mAP 60.3, FPS 5.0
VoD	SGDet3D: EAA 59.75, RoI 77.42	EAA 62.37, RoI 82.25

5. Route-constrained distillation as an earlier RCTDistill usage

In the earlier route-constrained usage, distillation is framed as curriculum learning over the teacher’s optimization route rather than as cross-modal BEV supervision (Jin et al., 2019). The teacher trajectory is represented by checkpoints $\hat{B}^{\text{RC}}_{\text{low}}$7, called anchor points. The student is trained sequentially against these anchor points so that it first learns from earlier, less deterministic teacher states and only later from stronger, converged states.

Two anchor-selection strategies are described. Equal Epoch Interval samples checkpoints at fixed epoch intervals. Greedy Search uses a KL-divergence hardness measure on a validation set,

$\hat{B}^{\text{RC}}_{\text{low}}$8

and if $\hat{B}^{\text{RC}}_{\text{low}}$9, epoch $B^{\text{RC}}_{\text{high}}$0 is considered too hard and $B^{\text{RC}}_{\text{high}}$1 is selected as the next anchor (Jin et al., 2019). The stage-wise loss is

$B^{\text{RC}}_{\text{high}}$2

with analogous substitutions for hint or mimicking losses. Standard KD is thus replaced by a multi-anchor sequence in which the student follows the route of the teacher through parameter space.

Empirically, this route-constrained formulation improves over conventional KD on close-set classification and open-set face recognition. On CIFAR-100 with teacher ResNet-50 and student MobileNetV2, Student-KD reaches 68.71% top-1 while Student-RCO reaches 70.85%, a gain of 2.14%. On ImageNet-1K, Student-KD reaches 66.75 top-1 and 87.3 top-5, while Student-RCO reaches 68.21 top-1 and 88.04 top-5. On MegaFace with feature-hint distillation, the route-constrained variant reaches 84.3% at $B^{\text{RC}}_{\text{high}}$3 distractors, improving over FitNet’s 81.02% (Jin et al., 2019). These results define a distinct meaning of RCTDistill: not cross-modal temporal fusion, but curriculum over teacher checkpoints.

6. Limitations, trade-offs, and broader significance

The 2025 cross-modal RCTDistill has explicit limitations. It depends on LiDAR during training, so fleets without LiDAR cannot directly train the method. Its motion modeling is simplified: TKD approximates trajectory regions with elliptical masks based on current velocity and fixed $B^{\text{RC}}_{\text{high}}$4, which may not capture extremely fast or complex motion. Its gains are bounded by teacher quality, and in scenes with very sparse radar or severe sensor failures, KD can only partially compensate (Bang et al., 22 Sep 2025).

The route-constrained usage has a different cost profile. It requires storing multiple teacher checkpoints and, in multi-stage form, can multiply training epochs. The paper proposes one-stage EEI to mitigate this, but anchor selection, threshold $B^{\text{RC}}_{\text{high}}$5, and the need for a full training trajectory remain practical constraints (Jin et al., 2019).

Taken together, these usages place RCTDistill within a broader distillation pattern in which the teacher supplies more than final logits alone. In the radar-camera framework, the teacher supplies spatially anisotropic, temporally aligned, and relational BEV supervision (Bang et al., 22 Sep 2025). In the route-constrained framework, the teacher supplies a sequence of progressively harder targets along its optimization path (Jin et al., 2019). A plausible implication is that the shared label has come to denote distillation schemes that emphasize structured teacher guidance—through space, time, or training trajectory—rather than a single canonical algorithm.