ECDet: Compact ViT Object Detector

• ECDet is a compact Vision Transformer-based detector designed for resource-constrained devices, integrating a distilled backbone and a streamlined encoder-decoder architecture.
• It employs lightweight multi-scale feature generation and an RT-DETR-style encoder to enhance dense prediction accuracy while maintaining efficiency.
• ECDet underpins EdgeCrafter by serving as the representation-learning anchor, with its distilled features being transferred to tasks like instance segmentation and human pose estimation.

ECDet is the compact Vision Transformer (ViT)-based object detector introduced as the core detection model of EdgeCrafter, a unified framework for edge dense prediction on resource-constrained devices (Liu et al., 19 Mar 2026). It is designed for lightweight object detection with a distilled compact backbone, an RT-DETR-style encoder-decoder, and a task-specialized distillation pipeline from a detection-adapted DINOv3 teacher. Within EdgeCrafter, ECDet also serves as the representation-learning anchor for the broader family, since the same detection-distilled backbone and encoder are transferred to ECInsSeg for instance segmentation and ECPose for human pose estimation (Liu et al., 19 Mar 2026).

1. Definition and design thesis

ECDet targets the regime in which computation, memory, and model size are tightly constrained, but dense prediction accuracy remains critical. The motivating claim of EdgeCrafter is that compact ViTs are not intrinsically mismatched to edge dense prediction; rather, their typical underperformance is attributed to insufficient task-specialized representation learning in small-scale ViTs. ECDet operationalizes that claim through a detector that couples a compact ViT backbone, lightweight multi-scale feature generation, an edge-friendly encoder-decoder, and a detection-specific distillation stage (Liu et al., 19 Mar 2026).

In this formulation, ECDet is not merely a detector head attached to a generic backbone. Its backbone, encoder, and training recipe are co-designed. The detector is therefore best understood as a compact ViT detection system whose central novelty lies as much in representation transfer as in architectural composition. This also explains its role inside EdgeCrafter: the detector is the primary vehicle through which the distilled representation is learned before being reused by the segmentation and pose branches.

2. Architectural composition

ECDet combines four principal components: ECViT as the compact backbone, a lightweight multi-scale feature generator, an RT-DETR-style encoder, and a DETR-style decoder with fixed query count (Liu et al., 19 Mar 2026).

The backbone replaces standard single-step patch embedding with a four-layer convolutional stem composed of four $3\times 3$ convolutions, each with stride $2$, so the effective stride remains $16$ while local structure is preserved more gradually. Four backbone scales are defined: $T$, $T+$, $S$, and $S+$. These are used respectively in ECDet-S, ECDet-M, ECDet-L, and ECDet-X, with scale variation mainly through embedding dimension, attention heads, and FFN ratio. The reported progression is embed dimension $192 \rightarrow 384$, attention heads $3 \rightarrow 6$, and FFN ratio $4 \rightarrow 6$.

Because the backbone is not hierarchical, ECDet explicitly constructs a three-level pyramid from late transformer tokens. If $2$0 and $2$1 are the outputs of the last two blocks, the base stride-16 representation is

$2$2

Features at strides $2$3, $2$4, and $2$5 are then produced by bilinear resizing and $2$6 projection:

$2$7

This is deliberately lighter than a conventional FPN-style neck. In ablation, simple mean fusion over the last two layers was retained because it matched or exceeded more elaborate alternatives after accounting for cost.

The encoder follows the RT-DETR pattern. The coarsest feature is first refined by AIFI,

$2$8

and then fused with the finer scales through CCFF,

$2$9

The decoder uses the standard DETR set-prediction paradigm with learned object queries $16$0, self-attention, deformable cross-attention, and FFN layers. ECDet fixes $16$1 decoder layers and $16$2 object queries for all scales. The detection head is the standard RT-DETR / DETR-style head attached to decoder outputs.

3. Task-specialized distillation

The defining training mechanism is task-specialized distillation from a detection-adapted DINOv3 teacher rather than from a generic pretrained backbone (Liu et al., 19 Mar 2026).

The teacher is first adapted to object detection using the ECDet-style detector formulation, producing ECTeacher. Two teacher scales are used: ECTeacher-S from DINOv3-S, and ECTeacher-B from DINOv3-B. ECDet-S uses ECTeacher-S, whereas ECDet-M, ECDet-L, and ECDet-X use ECTeacher-B. The paper emphasizes that teacher capacity must remain matched to student scale: for ECViT-T+ evaluated through ECDet-M, a DINOv3-B teacher yielded $16$3 AP, whereas DINOv3-L dropped to $16$4, which the authors interpret as evidence of an unbridgeable representation gap when the teacher is too large.

Distillation is performed at the feature level. Let $16$5 denote the student’s final-layer token features, $16$6 and $16$7 the teacher’s last two layers, and $16$8 a learned linear adapter from student to teacher dimension. The distillation objective is

$16$9

No separate prediction-level student-teacher KD term is reported for ECDet itself. The teacher is trained for detection first; the student backbone is then distilled through feature alignment and later used in normal detector training. The paper identifies this as the principal mechanism by which compact ViTs acquire localization-sensitive dense-prediction features.

Several ablations reinforce that interpretation. For ECViT-T+ evaluated via ECDet-M, adapting the teacher to COCO detection improved downstream AP from $T$0 to $T$1. Using ImageNet-1K plus COCO images for distillation slightly improved over ImageNet-1K alone, $T$2 versus $T$3. Aligning one student late layer to the teacher’s last two layers was adopted as the final recipe for robustness across scales, although a last-three-layer alignment reached $T$4 AP in the specific ECViT-T+ ablation. During distillation, LARS outperformed AdamW, and one register token was best among the tested settings.

4. Detection objective and optimization

After distillation, ECDet is trained on COCO with a DETR-style objective augmented by D-FINE localization terms (Liu et al., 19 Mar 2026). The detection loss is

$T$5

The reported weights are $T$6, $T$7, $T$8, $T$9, and $T+$0. The paper does not provide the explicit Hungarian matching cost.

Distillation pretraining uses ImageNet-1K and COCO train images for $T+$1 epochs with LARS, $T+$2 epochs of linear warmup, cosine decay to $T+$3 of peak LR, batch size $T+$4, weight decay $T+$5, and image size $T+$6. The peak learning rate follows

$T+$7

with base LR $T+$8 for ECViT-T/T+ and $T+$9 for ECViT-S/S+.

Detector training uses COCO train2017 for training and COCO val2017 for evaluation, with input resolution $S$0, AdamW, total batch size $S$1, and a progressive schedule ending with $S$2 epochs without heavy augmentation. Mosaic and Mixup are used in the first half of training, with probability $S$3 for S/M and $S$4 for L/X. Training durations are $S$5 epochs for S, $S$6 for M, and $S$7 for both L and X. The system does not rely on Objects365 pretraining; the external prior enters through the DINOv3-based teacher, which is itself adapted to COCO detection.

5. Performance, scaling, and ablations

On COCO val2017, ECDet forms a scale-consistent detector family with increasing AP, parameter count, FLOPs, and latency (Liu et al., 19 Mar 2026).

Variant	Params / FLOPs / Latency	COCO AP
ECDet-S	10M / 26G / 5.41 ms	51.7
ECDet-M	18M / 53G / 7.98 ms	54.3
ECDet-L	31M / 101G / 10.49 ms	57.0
ECDet-X	49M / 151G / 12.70 ms	57.9

Latency is measured on NVIDIA T4 with batch size $S$8, FP16, and TensorRT v10.6. The abstract highlights ECDet-S at $S$9 AP with fewer than $S+$0M parameters; the main table reports $S+$1M parameters, $S+$2 GFLOPs, and $S+$3 ms latency.

The reported comparisons position ECDet as strongest among methods trained only on COCO at several scales. ECDet-S exceeds YOLOv9-S, YOLOv10-S, YOLO11-S, YOLOv12-S-turbo, RT-DETRv2-S, D-FINE-S, DEIM-S, DEIMv2-S, and RT-DETRv4-S, reaching $S+$4 AP versus $S+$5 for DEIMv2-S. ECDet-M reaches $S+$6 AP, above RT-DETRv4-M at $S+$7 and DEIMv2-M at $S+$8. ECDet-L reaches $S+$9 AP, above RT-DETRv4-L at $192 \rightarrow 384$0, DEIMv2-L at $192 \rightarrow 384$1, and LW-DETR-L at $192 \rightarrow 384$2 despite the latter’s Objects365 pretraining. ECDet-X reaches $192 \rightarrow 384$3 AP, slightly above RT-DETRv4-X at $192 \rightarrow 384$4 and DEIMv2-X at $192 \rightarrow 384$5, while remaining below some much larger Objects365-pretrained systems.

The ablations attribute a substantial part of this performance to task-specialized representation learning. For ECDet-M, replacing the convolutional stem with vanilla patch embedding reduced AP from $192 \rightarrow 384$6 to $192 \rightarrow 384$7, with a marked drop in small-object performance from $192 \rightarrow 384$8 to $192 \rightarrow 384$9 on $3 \rightarrow 6$0. Mean fusion of the last two transformer blocks was selected because it achieved $3 \rightarrow 6$1 AP with lower cost than concatenation and comparable accuracy to STA fusion. The paper also reports approximate training cost for ECDet-M at $3 \rightarrow 6$2 GPU hours, compared with about $3 \rightarrow 6$3 for RT-DETRv4-M and $3 \rightarrow 6$4 for YOLO11-M.

6. Role in EdgeCrafter and limitations

Within EdgeCrafter, ECDet has a dual status: it is both the object detector and the representation-learning substrate for the broader edge dense prediction family (Liu et al., 19 Mar 2026). The same detection-distilled backbone and encoder are reused in ECInsSeg and ECPose, with lightweight task-specific heads added on top. This arrangement makes detection the pretext through which the compact ViT is specialized for downstream dense prediction.

The paper identifies several strengths. ECDet offers a strong AP-to-size tradeoff, especially at small and medium scales; it remains competitive without Objects365 pretraining; it shares a common representation across detection, segmentation, and pose; and it is benchmarked under a deployment-oriented latency protocol using TensorRT FP16 on T4. At the same time, the limitations are explicit. ECDet is not always the fastest latency-wise, as CNN and YOLO systems remain more software-optimized. It still depends on a powerful DINOv3-based teacher during pretraining. No quantization benchmarks, ONNX export details, mobile runtime results, or real embedded-device evaluations are reported. Memory usage is not explicitly measured, and the motivating discussion of low-TOPS SoCs such as Rockchip RK3568 is not accompanied by direct hardware benchmarks.

A plausible implication is that ECDet is most attractive where model size and detector accuracy matter more than the absolute minimum end-to-end latency, and where FP16/TensorRT transformer deployment is feasible. In that sense, ECDet is less a claim that compact ViTs universally dominate edge CNN detectors than a demonstration that, with detection-specialized distillation and a carefully simplified encoder-decoder, compact ViTs can be practical edge dense predictors rather than merely compressed classification backbones repurposed for detection.