---
title: 'L³U-net: Lightweight U-net for Segmentation'
url: https://www.emergentmind.com/topics/l-u-net
type: topic
---

# L³U-net: Lightweight U-net for Segmentation

L³U-net (Low-Latency Lightweight U-net Based Image Segmentation Model for Parallel CNN Processors) is a compact deep neural network architecture specifically designed for real-time image segmentation on resource-constrained edge devices. Characterized by a "tiny U-net" backbone and an advanced data-folding technique, L³U-net achieves efficient hardware utilization, low memory footprint, and high inference speeds on parallel CNN processors such as the MAX78000. The system features quantization-aware optimization, streamlined computational stages, and robust segmentation accuracy (>90% pixel accuracy) across diverse datasets, all within the constraints of battery-powered, low-SWaP (Size, Weight, and Power) TinyML deployments [2203.16528].

## 1. Network Architecture

L³U-net employs a "micro" U-net configuration, integrating a streamlined encoder–decoder topology with skip connections. Input RGB images of size $352 \times 352$ pixels are subjected to an initial data-folding operation with stride $\alpha = 4$, reshaping the tensor from $[3, 352, 352]$ to $[48, 88, 88]$. The architecture consists of the following pipeline:

- **Head:** Three sequential $1 \times 1$ convolutions (16, 32, 48 output channels), each followed by BN–ReLU, and a $3 \times 3$ convolution (48 channels).
- **Encoder:** Three levels with max-pooling (stride 2), each followed by two $3 \times 3$ convolutions and BN–ReLU activations. Channel progression: 64, 128, and 256 output channels.
- **Bottleneck:** Terminal convolutional block at $256 \times 11 \times 11$ spatial location.
- **Decoder:** Three-step upsampling via ConvTranspose2D (stride 2), with concatenation of corresponding encoder features and head output, followed by two $3 \times 3$ convolutions after each upsampling.
- **Tail:** Four $1 \times 1$ convolutions per class, BN–ReLU, then unfolding (inverse folding) and spatial upsampling to $352 \times 352$ final segmentation.

Skip connections are adopted as in conventional U-net designs, concatenating encoder outputs to decoder blocks at corresponding spatial resolutions. With a total parameter count of 278,176 (4-class) or 277,004 (2-class), the network footprint is minimal compared to typical segmentation CNNs.

## 2. Data-Folding Method for Parallelization

L³U-net's principal innovation is a data-folding transformation that exploits hardware parallelism by recasting high-resolution, low-channel image data into lower-resolution, high-channel inputs. Formally, an input tensor $X \in \mathbb{R}^{C \times H \times W}$ is folded to $X^f \in \mathbb{R}^{C \cdot \alpha^2 \times H/\alpha \times W/\alpha}$ by:

\[
X^f[ c \cdot \alpha^2 + i \cdot \alpha + j, h', w' ] = X[ c, \alpha h' + i, \alpha w' + j ]
\]

for $c \in [0, C-1]$, $i, j \in [0, \alpha-1]$, $h' \in [0, H/\alpha-1]$, $w' \in [0, W/\alpha-1]$.

This transformation ensures that a parallel accelerator with $P$ cores is optimally loaded, provided $C \cdot \alpha^2 \leq P$, eliminating idle compute and reducing inference latency. Convolutional kernels are similarly reshaped for mathematical equivalence in the folded domain, permitting conventional $k \times k$ convolutions with stride $s$ post-folding to match $ \alpha k \times \alpha k $ kernels with stride $ \alpha s $ in the unfolded domain.

## 3. Computational Complexity and Memory Considerations

Folding the spatial dimension into the channel dimension does not alter total MAC operations:

\[
\text{MACs}_{\text{unfolded}} = C \cdot H \cdot W \cdot k^2,\qquad
\text{MACs}_{\text{folded}} = (C \cdot \alpha^2) \cdot (H/\alpha) \cdot (W/\alpha) \cdot k^2 = C \cdot H \cdot W \cdot k^2
\]

However, per-inference latency is reduced by up to $1/\alpha^2$, contingent upon full core utilization. The activation memory footprint is pseudo-invariant, but per-core buffers for intermediate activations scale down by a factor of $\alpha$ in each spatial dimension. No persistent storage cost is incurred beyond the temporary reshape.

## 4. Deployment on Parallel CNN Accelerators

L³U-net was implemented and evaluated on the MAX78000 hardware platform, which integrates an Arm Cortex-M4 microcontroller unit and a 64-core CNN accelerator. Model adaptation for deployment includes:

- **Quantization-Aware Training (QAT):** Employs "fake quant" to enable 8-bit integer weights; BatchNorm is fused pre-QAT for deployment efficiency.
- **Core Mapping and Tiling:** Model is unrolled to maximize the use of available parallel MAC cores, leveraging the channel-inflated tensor structure produced by data folding.
- **Memory Usage:** The complete quantized model requires approximately 278 kB of SRAM, substantially below the 442 kB hardware limit.

On-device execution achieves single-image inference times of 90–95 ms (≈10 fps) with 6.9–7.3 mJ per inference, supporting inference rates in excess of 1.5 million executions per typical AA alkaline battery.

## 5. Empirical Segmentation Performance

L³U-net demonstrates high segmentation accuracy across standard datasets with minimal resource usage. Table 1 summarizes performance metrics:

| Dataset        | Pixel Accuracy (%) | mIoU (%) | Latency (ms) | Energy/Inf. (mJ) |
| -------------- | ----------------- | -------- | ------------ | ---------------- |
| CamVid (4-cl)  | 91.05             | 84.24    | 95.1         | 7.3              |
| AISegment (2-cl)| 99.19            | 98.09    | 90.3         | 6.9              |

Qualitatively, segmentation outputs on CamVid accurately delineate common classes (Building, Sky, Tree, Other), with the primary failure mode being thin-structure undersegmentation. On AISegment, binary matting of foreground portrait and background is consistently high-fidelity on $352 \times 352$ inputs.

Compared to edge-oriented architectures like AttendSeg and EdgeSegNet, L³U-net achieves 4×–30× fewer parameters, 10×–100× fewer MACs, and is uniquely realized on a battery-powered, 8-bit integer accelerator (contrasting with floating-point or FPGA-based alternatives).

## 6. Limitations and Future Application Directions

While L³U-net attains close to state-of-the-art U-net segmentation accuracy under stringent energy and latency constraints, architectural limits are imposed by the scope of the data-folding operation:

- **Current Restriction:** Only the head layer is folded given MAX78000 memory/reshape limitations.
- **Potential Extensions:** Folding deeper layers and enabling dynamic per-layer folding factors ($\alpha$) could further exploit core utilization but would necessitate adaptable accelerator memory mapping.
- **Limitations:** Very small or thin object classes are less reliably segmented given the aggressive folding; this suggests that finer-grain folding or architectural enhancements may improve granularity.
- **Broader Utility:** Data-folding as a technique may generalize to other multi-core or batched accelerator targets for efficient low-latency inference.

In summary, L³U-net's coupling of a micro-U-net core and data-folding frontend enables $>$90% mIoU segmentation in $\approx$100 ms on a $<1$ mW-class parallel CNN accelerator [2203.16528].

Source: https://www.emergentmind.com/topics/l-u-net