L³U-net: Lightweight U-net for Segmentation

• L³U-net is a compact deep neural network architecture that leverages a micro U-net backbone and innovative data-folding to enable efficient segmentation on edge devices.
• It employs a streamlined encoder–decoder design with quantization-aware training, achieving high pixel accuracy (>90%) while significantly reducing computational costs.
• The system is optimized for parallel CNN processors like the MAX78000, delivering low-latency inference (~95 ms per image) with minimal power and memory usage.

L³U-net (Low-Latency Lightweight U-net Based Image Segmentation Model for Parallel CNN Processors) is a compact deep neural network architecture specifically designed for real-time image segmentation on resource-constrained edge devices. Characterized by a "tiny U-net" backbone and an advanced data-folding technique, L³U-net achieves efficient hardware utilization, low memory footprint, and high inference speeds on parallel CNN processors such as the MAX78000. The system features quantization-aware optimization, streamlined computational stages, and robust segmentation accuracy (>90% pixel accuracy) across diverse datasets, all within the constraints of battery-powered, low-SWaP (Size, Weight, and Power) TinyML deployments (Okman et al., 2022).

1. Network Architecture

L³U-net employs a "micro" U-net configuration, integrating a streamlined encoder–decoder topology with skip connections. Input RGB images of size $352 \times 352$ pixels are subjected to an initial data-folding operation with stride $\alpha = 4$, reshaping the tensor from $[3, 352, 352]$ to $[48, 88, 88]$. The architecture consists of the following pipeline:

• Head: Three sequential $1 \times 1$ convolutions (16, 32, 48 output channels), each followed by BN–ReLU, and a $3 \times 3$ convolution (48 channels).
• Encoder: Three levels with max-pooling (stride 2), each followed by two $3 \times 3$ convolutions and BN–ReLU activations. Channel progression: 64, 128, and 256 output channels.
• Bottleneck: Terminal convolutional block at $256 \times 11 \times 11$ spatial location.
• Decoder: Three-step upsampling via ConvTranspose2D (stride 2), with concatenation of corresponding encoder features and head output, followed by two $3 \times 3$ convolutions after each upsampling.
• Tail: Four $1 \times 1$ convolutions per class, BN–ReLU, then unfolding (inverse folding) and spatial upsampling to $\alpha = 4$0 final segmentation.

Skip connections are adopted as in conventional U-net designs, concatenating encoder outputs to decoder blocks at corresponding spatial resolutions. With a total parameter count of 278,176 (4-class) or 277,004 (2-class), the network footprint is minimal compared to typical segmentation CNNs.

2. Data-Folding Method for Parallelization

L³U-net's principal innovation is a data-folding transformation that exploits hardware parallelism by recasting high-resolution, low-channel image data into lower-resolution, high-channel inputs. Formally, an input tensor $\alpha = 4$1 is folded to $\alpha = 4$2 by:

$\alpha = 4$3

for $\alpha = 4$4, $\alpha = 4$5, $\alpha = 4$6, $\alpha = 4$7.

This transformation ensures that a parallel accelerator with $\alpha = 4$8 cores is optimally loaded, provided $\alpha = 4$9, eliminating idle compute and reducing inference latency. Convolutional kernels are similarly reshaped for mathematical equivalence in the folded domain, permitting conventional $[3, 352, 352]$0 convolutions with stride $[3, 352, 352]$1 post-folding to match $[3, 352, 352]$2 kernels with stride $[3, 352, 352]$3 in the unfolded domain.

3. Computational Complexity and Memory Considerations

Folding the spatial dimension into the channel dimension does not alter total MAC operations:

$[3, 352, 352]$4

However, per-inference latency is reduced by up to $[3, 352, 352]$5, contingent upon full core utilization. The activation memory footprint is pseudo-invariant, but per-core buffers for intermediate activations scale down by a factor of $[3, 352, 352]$6 in each spatial dimension. No persistent storage cost is incurred beyond the temporary reshape.

4. Deployment on Parallel CNN Accelerators

L³U-net was implemented and evaluated on the MAX78000 hardware platform, which integrates an Arm Cortex-M4 microcontroller unit and a 64-core CNN accelerator. Model adaptation for deployment includes:

• Quantization-Aware Training (QAT): Employs "fake quant" to enable 8-bit integer weights; BatchNorm is fused pre-QAT for deployment efficiency.
• Core Mapping and Tiling: Model is unrolled to maximize the use of available parallel MAC cores, leveraging the channel-inflated tensor structure produced by data folding.
• Memory Usage: The complete quantized model requires approximately 278 kB of SRAM, substantially below the 442 kB hardware limit.

On-device execution achieves single-image inference times of 90–95 ms (≈10 fps) with 6.9–7.3 mJ per inference, supporting inference rates in excess of 1.5 million executions per typical AA alkaline battery.

5. Empirical Segmentation Performance

L³U-net demonstrates high segmentation accuracy across standard datasets with minimal resource usage. Table 1 summarizes performance metrics:

Dataset	Pixel Accuracy (%)	mIoU (%)	Latency (ms)	Energy/Inf. (mJ)
CamVid (4-cl)	91.05	84.24	95.1	7.3
AISegment (2-cl)	99.19	98.09	90.3	6.9

Qualitatively, segmentation outputs on CamVid accurately delineate common classes (Building, Sky, Tree, Other), with the primary failure mode being thin-structure undersegmentation. On AISegment, binary matting of foreground portrait and background is consistently high-fidelity on $[3, 352, 352]$7 inputs.

Compared to edge-oriented architectures like AttendSeg and EdgeSegNet, L³U-net achieves 4×–30× fewer parameters, 10×–100× fewer MACs, and is uniquely realized on a battery-powered, 8-bit integer accelerator (contrasting with floating-point or FPGA-based alternatives).

6. Limitations and Future Application Directions

While L³U-net attains close to state-of-the-art U-net segmentation accuracy under stringent energy and latency constraints, architectural limits are imposed by the scope of the data-folding operation:

• Current Restriction: Only the head layer is folded given MAX78000 memory/reshape limitations.
• Potential Extensions: Folding deeper layers and enabling dynamic per-layer folding factors ($[3, 352, 352]$8) could further exploit core utilization but would necessitate adaptable accelerator memory mapping.
• Limitations: Very small or thin object classes are less reliably segmented given the aggressive folding; this suggests that finer-grain folding or architectural enhancements may improve granularity.
• Broader Utility: Data-folding as a technique may generalize to other multi-core or batched accelerator targets for efficient low-latency inference.

In summary, L³U-net's coupling of a micro-U-net core and data-folding frontend enables $[3, 352, 352]$990% mIoU segmentation in $[48, 88, 88]$0100 ms on a $[48, 88, 88]$1 mW-class parallel CNN accelerator (Okman et al., 2022).