TriLiteNet: Lightweight Multi-Task Perception

• TriLiteNet is a lightweight, unified encoder–decoder network enabling efficient multi-task visual perception by simultaneously performing vehicle detection, drivable area segmentation, and lane line segmentation.
• It leverages a modified ESPNet backbone with depthwise dilated separable convolutions and simple, task-specific decoders to minimize computational cost and memory usage for embedded deployment.
• In BDD100K experiments, TriLiteNet variants achieve competitive segmentation metrics while trading slight detection performance for a substantial reduction in parameters and FLOPs.

Searching arXiv for TriLiteNet and closely related papers to ground the article with citations. arXiv search query: "TriLiteNet: Lightweight Model for Multi-Task Visual Perception" arXiv search query: "YOLOP panoramic driving perception" TriLiteNet is a lightweight, unified encoder–decoder network for multi-task visual perception in Advanced Driver Assistance Systems and autonomous driving. It is designed to perform three camera-based tasks simultaneously—vehicle detection, drivable area segmentation, and lane line segmentation—while maintaining low computational cost and embedded deployability. In the reported BDD100K experiments, the largest configuration, TriLiteNet$_{base}$, attains 85.6\% recall for vehicle detection, 92.4\% mIoU for drivable area segmentation, and 82.3\% balanced accuracy for lane line segmentation with 2.35M parameters and 7.72 GFLOPs; the architecture also includes a tiny variant with 0.15M parameters for more constrained settings (Che et al., 4 Sep 2025).

1. Problem setting and design criteria

TriLiteNet is motivated by panoramic driving perception in ADAS, where a single camera pipeline is expected to determine what objects are present, where the ego vehicle can safely drive, and where lane boundaries lie. In the formulation used here, this panoramic understanding is decomposed into vehicle detection, drivable area segmentation, and lane line segmentation. A system that performs these three tasks jointly can support collision avoidance, path planning, and lane keeping from a single camera (Che et al., 4 Sep 2025).

The model is explicitly positioned against a common deployment problem: single-task systems typically maintain separate encoders or backbones for detection and segmentation, so running three independent networks multiplies FLOPs, memory, and latency. The paper frames multi-task learning as a remedy because a shared encoder with task-specific heads reduces redundancy and can improve generalization through shared features. Existing multi-task driving-perception systems such as YOLOP, YOLOPv2/v3, YOLOPX, A-YOLOM, HybridNets, and U-PDP are described as demonstrating the viability of this approach, but the stated limitations are heavy architectures, detection-centric backbones that are not ideal for segmentation efficiency, and reporting practices centered on powerful GPUs rather than embedded devices (Che et al., 4 Sep 2025).

The design targets are therefore operational rather than purely benchmark-driven: low FLOPs and parameter count, real-time latency on Jetson-class hardware, low power consumption, and competitive accuracy on BDD100K. The stated target regime includes sub-10 GFLOPs and roughly 1–3M parameters, with an even smaller sub-0.2M operating point for extreme setups. This deployment orientation is central to the model’s identity: TriLiteNet is not presented as a maximal-accuracy detector, but as a multi-task network engineered for resource-constrained embedded ADAS.

2. Network architecture

TriLiteNet uses a unified encoder–decoder topology with a shared lightweight backbone and three task-specific outputs: a detection decoder, a drivable area segmentation head, and a lane line segmentation head. The encoder is derived from ESPNet and is modified through the replacement of standard dilated convolutions with depthwise dilated separable convolutions, producing what the paper calls the Depth-wise ESP block (Che et al., 4 Sep 2025).

The backbone retains the classic ESP logic—channel reduction through a $1 \times 1$ convolution, split-and-transform through multiple parallel dilated convolutions, and merge through hierarchical feature fusion and a skip connection—but substitutes the transform stage with depthwise dilated separable convolutions. The stated rationale is to preserve large receptive fields while reducing FLOPs and parameter count. The encoder outputs five feature scales: $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$ The addition of $C_4$ and $C_5$ distinguishes the design from earlier ESPNet-based work such as TwinLiteNet, which stopped at $1/8$ resolution; the paper identifies these deeper scales as crucial for multi-scale object detection.

Two downsampled RGB inputs are also constructed through average pooling,

$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$

and are injected into the segmentation decoders through skip connections. This design choice supplies low-resolution raw image content to the decoders.

The detection branch takes $C_3$, $C_4$, and $C_5$ as input. It applies Spatial Pyramid Pooling on $1 \times 1$0 to enrich global context and scale robustness, then processes the resulting multiscale features with LitePAN, a lightweight Path Aggregation Network variant using $1 \times 1$1 convolutions for channel alignment, depthwise separable convolutions for transformation, and element-wise addition for same-resolution fusion. The neck outputs $1 \times 1$2, $1 \times 1$3, and $1 \times 1$4 at $1 \times 1$5, $1 \times 1$6, and $1 \times 1$7 resolution, respectively. The final detection head is anchor-based rather than anchor-free, uses auto-anchor via K-means as in YOLOv4, assigns three anchors per grid cell at each scale, and predicts classification logits, objectness, and bounding box offsets and sizes (Che et al., 4 Sep 2025).

The segmentation side comprises two structurally identical but parametrically separate heads, one for drivable area and one for lane lines. Both use only $1 \times 1$8 as the main encoder feature, passing it through Partial Class Activation Attention (PCAA) to obtain $1 \times 1$9. Each head then applies transposed convolutions for upsampling, standard convolutions for refinement, and skip concatenations with $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$0 at $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$1 scale and $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$2 at $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$3 scale. The outputs are intermediate tensors $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$4 at $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$5, $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$6 at $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$7, and final binary logits

$$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$8

The paper emphasizes that these decoders are intentionally simple and omit heavier modules such as ASPP or transformer-based decoders.

All three published variants share the same topology and differ only in channel width and repetition counts $$C_1 \in \mathbb{R}^{H/2 \times W/2}, \quad C_2 \in \mathbb{R}^{H/4 \times W/4}, \quad C_3 \in \mathbb{R}^{H/8 \times W/8}, \quad C_4 \in \mathbb{R}^{H/16 \times W/16}, \quad C_5 \in \mathbb{R}^{H/32 \times W/32}.$$9. TriLiteNet$C_4$0 uses $C_4$1, 0.15M parameters, and 0.55G FLOPs; TriLiteNet$C_4$2 uses $C_4$3, 0.59M parameters, and 1.99G FLOPs; TriLiteNet$C_4$4 uses $C_4$5, 2.35M parameters, and 7.72G FLOPs (Che et al., 4 Sep 2025).

3. Optimization, loss formulation, and training protocol

TriLiteNet is trained with a task-weighted multi-task objective,

$C_4$6

with $C_4$7, $C_4$8, and $C_4$9 (Che et al., 4 Sep 2025).

The detection loss is decomposed as

$C_5$0

where $C_5$1 is a BCE loss for class prediction, $C_5$2 is a BCE loss for objectness, and $C_5$3 is an IoU-based bounding-box regression loss. The reported weights are $C_5$4, $C_5$5, and $C_5$6, borrowed from YOLOP tuning.

For each segmentation task $C_5$7, the loss is

$C_5$8

The focal loss uses $C_5$9 and $1/8$0 for both tasks. The Tversky component is chosen to handle class imbalance by weighting false positives and false negatives asymmetrically: $1/8$1 with loss $1/8$2. The reported hyperparameters are $1/8$3 for drivable area and $1/8$4 for lane lines, the latter intended to penalize false negatives more heavily because lane markings are thin targets. The paper explicitly describes this focal-plus-Tversky combination as particularly well suited to severely imbalanced segmentation tasks, especially lane segmentation (Che et al., 4 Sep 2025).

Training is performed in PyTorch with AdamW, an initial learning rate of 0.001, $1/8$5, $1/8$6, warm-up followed by cosine annealing, 200 epochs, and batch size 16. Training is from scratch, with no ImageNet or other pretraining. The dataset is BDD100K, using the official splits of 70k training images, 10k validation images, and 20k test images, though evaluation is reported on validation because the test labels are not public. Images are resized from the original 1280$1/8$7720 to 640$1/8$8384 (Che et al., 4 Sep 2025).

The label protocol is task-specific. Detection merges car, truck, bus, and train into a single “vehicle” class. Drivable area merges “direct drivable area” and “alternative drivable area” into one drivable region. Lane labels are thickened to 8 pixels during training while validation uses the standard 2-pixel width, following the established practice cited in the paper for YOLOP and ENet-SAD–style training. The training loop maintains an Exponential Moving Average shadow model, and the final EMA weights are used for evaluation and inference; the paper reports that ablation shows consistent improvement across tasks from EMA.

4. Quantitative behavior

The reported evaluation metrics are Recall and [email protected] for detection, mIoU for drivable area segmentation, and both Balanced Accuracy and IoU for lane line segmentation. Under this protocol, TriLiteNet$1/8$9 reaches 76.5\% recall, 49.6\% [email protected], 88.5\% drivable mIoU, 75.6\% lane accuracy, and 24.2\% lane IoU. TriLiteNet$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$0 reaches 81.6\% recall, 63.2\% [email protected], 91.0\% drivable mIoU, 79.5\% lane accuracy, and 27.6\% lane IoU. TriLiteNet$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$1 reaches 85.6\% recall, 72.3\% [email protected], 92.4\% drivable mIoU, 82.3\% lane accuracy, and 29.8\% lane IoU (Che et al., 4 Sep 2025).

The paper also reports direct efficiency comparisons against several multi-task baselines at 640$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$2384. In terms of model size and FLOPs, TriLiteNet$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$3 is 2.35M parameters and 7.72G FLOPs, compared with YOLOP at 7.9M and 9.38G, YOLOPv2 at 38.9M and 55G, YOLOPv3 at 30.2M and 35.38G, HybridNets at 13.8M and 25.69G, YOLOPX at 32.9M and 44.86G, A-YOLOM(n) at 4.43M and 6.66G, and A-YOLOM(s) at 13.61M and 19.22G. The paper summarizes this by stating that TriLiteNet$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$4 is approximately $$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$5 smaller in parameters and slightly lighter in FLOPs than YOLOP, and over an order of magnitude smaller than YOLOPv2, YOLOPv3, and YOLOPX (Che et al., 4 Sep 2025).

At the level of task accuracy relative to these baselines, the paper characterizes the outcome as a deliberate trade-off. TriLiteNet$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$6 reports lower detection metrics than the largest YOLO-based multi-task models, whose recall values lie in the reported $$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$7–$$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$8\% range and [email protected] values exceed $$I_1 \in \mathbb{R}^{3 \times H/2 \times W/2}, \qquad I_2 \in \mathbb{R}^{3 \times H/4 \times W/4},$$9\%. By contrast, drivable area segmentation is near the top of the reported range at 92.4\% mIoU, exceeding YOLOP and both A-YOLOM variants while remaining slightly below YOLOPv2, YOLOPX, and YOLOPv3 at 93.2\%. Lane IoU at 29.8\% exceeds YOLOP and both A-YOLOM variants and is described as comparable with HybridNets at 31.6\%, despite HybridNets being approximately $C_3$0–$C_3$1 larger. The paper’s own summary is that TriLiteNet trades some detection performance for considerably lower compute and very strong segmentation performance.

A separate ablation compares single-task and joint training for TriLiteNet$C_3$2. The single-task detector uses 0.47M parameters and 1.18G FLOPs, with 81.3\% recall and 63.3\% [email protected]. The drivable-area-only model uses 0.18M parameters and 1.47G FLOPs, with 90.8\% mIoU. The lane-only model also uses 0.18M parameters and 1.47G FLOPs, with 75.8\% accuracy and 29.3\% IoU. The joint TriLiteNet$C_3$3 model uses 0.59M parameters and 1.99G FLOPs, attaining 81.6\% recall, 63.2\% [email protected], 91.0\% drivable mIoU, 79.5\% lane accuracy, and 27.6\% lane IoU. The paper concludes that multi-task learning preserves nearly the same task-specific performance, improves some metrics, and is computationally more efficient than three separate networks (Che et al., 4 Sep 2025).

5. Embedded deployment characteristics

A defining feature of TriLiteNet is that latency and power are reported on embedded platforms rather than only on desktop GPUs. The evaluation uses NVIDIA Jetson Xavier and Jetson TX2 with TensorRT FP16 (Che et al., 4 Sep 2025).

On Jetson Xavier, latency at batch size 1 is reported as $C_3$4 ms for TriLiteNet$C_3$5, $C_3$6 ms for TriLiteNet$C_3$7, and $C_3$8 ms for TriLiteNet$C_3$9. On Jetson TX2, the corresponding latencies are $C_4$0 ms, $C_4$1 ms, and $C_4$2 ms. The paper translates these measurements into approximate frame rates of about 111 FPS, 60 FPS, and 39 FPS on Xavier, and about 43 FPS, 26 FPS, and 21 FPS on TX2 for the three configurations, respectively. The stated interpretation is that all configurations deliver real-time performance on both platforms, especially Xavier.

Average power consumption is also reported. On Xavier, TriLiteNet$C_4$3 consumes $C_4$4 W, TriLiteNet$C_4$5 $C_4$6 W, and TriLiteNet$C_4$7 $C_4$8 W. On TX2, the corresponding values are $C_4$9 W, $C_5$0 W, and $C_5$1 W. The paper identifies the largest configuration as remaining below 23 W on Xavier and below 5 W on TX2, and emphasizes the tiny model as particularly power-efficient. A plausible implication is that the deployment study is intended to distinguish TriLiteNet from multi-task systems whose published throughput figures are concentrated on RTX 3090 or V100-class hardware.

The paper also reports desktop throughput for context. On RTX 4090 at 640$C_5$2384, TriLiteNet$C_5$3 reaches 185 FPS at batch size 1, 1340 FPS at batch size 8, and 3397 FPS at batch size 32; TriLiteNet$C_5$4 reaches 151, 1081, and 1641 FPS; TriLiteNet$C_5$5 reaches 105, 727, and 760 FPS. TriLiteNet$C_5$6 is reported as competitive with or faster than YOLOP across all listed batch sizes, while the tiny configuration is markedly faster than all listed baselines (Che et al., 4 Sep 2025).

6. Literature position, qualitative behavior, and limitations

TriLiteNet is situated within a line of multi-task driving-perception systems that share a single backbone across detection and segmentation tasks, but its architectural orientation differs from the detection-centric design of YOLO-style systems. The paper characterizes YOLOP, YOLOPv2, YOLOPv3, YOLOPX, and A-YOLOM as models that prioritize detection performance through heavier backbones and necks, whereas TriLiteNet uses a segmentation-oriented ESPNet-derived encoder and prioritizes extreme efficiency together with strong segmentation quality. It is also framed as an extension of TwinLiteNet and TwinLiteNetPlus, which addressed only the two segmentation tasks and required a separate detector (Che et al., 4 Sep 2025).

The qualitative evaluation covers variations in time of day, weather, and scene type, including daytime, dawn or dusk, night, snowy, rainy, overcast, foggy, tunnel, parking lot, gas station, and residential scenes. In these examples, TriLiteNet$C_5$7 is described as producing consistent drivable-area masks, robust lane predictions under low light, adverse weather, and occlusion, and solid vehicle detection, while heavier models may capture slightly more small objects in extremely crowded scenes. The smaller variants show some degradation in detection, especially for small or distant vehicles, while segmentation remains strong but can be less precise in complex scenes. This suggests that the architectural budget is disproportionately protective of segmentation quality, which is consistent with the ESP-derived backbone and the PCAA-based segmentation heads.

The paper explicitly notes several limitations. First, TriLiteNet$C_5$8 does not match the highest detection performance of YOLOPv2, YOLOPv3, and YOLOPX; this is presented as a conscious trade-off to keep the architecture small and segmentation-driven. Second, extremely complex scenes, including dense traffic, heavy occlusion, and unusual weather, remain challenging. Third, the task scope is limited to three outputs—vehicle detection, drivable area segmentation, and lane line segmentation—and does not include traffic signs, traffic signals, instance segmentation, or full panoptic segmentation (Che et al., 4 Sep 2025).

Future directions are described in deployment-centric terms: improving robustness to variable workloads and dynamic real-world conditions, extending the framework to additional perception tasks while maintaining low resource usage, and pursuing further compression, quantization, and hardware-specific optimization beyond the initial FP16 and INT8 experiments. Within that framing, TriLiteNet can be understood as a carefully constrained multi-task perception model whose central contribution lies not in maximizing a single benchmark metric, but in balancing detection, drivable-area segmentation, lane segmentation, latency, and power within a unified lightweight system (Che et al., 4 Sep 2025).