---
title: Scale-Driven Decoder (SDD) in FlexDepth
url: https://www.emergentmind.com/topics/scale-driven-decoder-sdd
type: topic
---

# Scale-Driven Decoder (SDD) in FlexDepth

Searching arXiv for the primary paper and closely related uses of “SDD” to ground the article in current literature.
arXiv search query: 2607.00736
The **Scale-Driven Decoder (SDD)** is the decoder architecture introduced in FlexDepth, a family of self-supervised monocular depth estimation models for driving scenes. In that setting, SDD is the architectural core that removes pre-convolution, introduces dynamic upsampling, and selects decoder components as a function of overall model scale rather than fixing one decoder design across all capacity points. It operates within a two-stage static–dynamic decoupled training pipeline, but the decoder itself is unchanged across stages; what varies is the supervision. The design is motivated by the combination of highly multi-scale geometry, dynamic traffic participants, and structure-sensitive depth discontinuities in road environments [2607.00736].

## 1. Design rationale and problem setting

Self-supervised monocular depth estimation in driving typically relies on monocular videos or stereo pairs together with photometric consistency, without ground-truth depth. The driving setting is described as highly multi-scale, dynamic, and structure-sensitive: very near road surfaces, mid-range vehicles and pedestrians, and far buildings or trees coexist; moving vehicles, cyclists, and pedestrians violate the static-scene assumption behind photometric loss; and safety depends on sharp object boundaries and correct depth for small or distant actors [2607.00736].

Most prior self-supervised MDE decoders are described as simple U-Net–style modules in which each scale follows
\[
\text{pre-conv} \rightarrow \text{upsample} \rightarrow \text{concat} \rightarrow \text{post-conv}.
\]
In the FlexDepth formulation, that pattern is problematic when the same decoder is scaled from tiny to large models. For lightweight models in particular, pre-convolution on low-resolution deep features is said to save FLOPs but to destroy already sparse global semantic information before spatial restoration, leading to blurred object boundaries, missing small or distant objects, and poor behavior when the same decoder design is reused across scales [2607.00736].

SDD is presented as the answer to that failure mode. Its defining commitments are architectural rather than loss-specific: it removes pre-conv entirely to preserve global semantics; it uses content-adaptive dynamic upsampling instead of fixed bilinear interpolation; and it changes the post-convolution block and prediction-head ordering according to model size. This design supports a five-model family—Nano, Small, Medium, Large, and X-Large—rather than a single capacity point [2607.00736].

## 2. Position within FlexDepth

FlexDepth takes an input image \(I_t \in \mathbb{R}^{H \times W \times 3}\) and passes it through a YOLOv11-based depth encoder, the SDD, a ResNet-18 pose network with a small decoder, and a self-supervised warping stage. The encoder uses “11n/s/m/l-seg” variants and produces multi-scale features \(\mathbf{F}_{enc,i}\) at resolutions \(\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}\) of the input. SDD then performs top–down decoding from the coarsest feature [2607.00736].

At decoder level \(i\), SDD receives a decoder feature \(\mathbf{F}_{i+1}\) and the corresponding encoder skip feature \(\mathbf{F}_{enc,i}\). The decoding step is
\[
D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},
\]
where \(\mathcal{U}\) is a scale-dependent upsampling operator, \(\mathcal{C}\) is the post-convolution block, and \(\mathcal{P}\) is the prediction head. The output is a refined decoder feature for the next higher resolution and, optionally, a depth prediction at scale \(i\) [2607.00736].

The decoder is “scale-driven” because its internal configuration depends on the overall model scale. The paper specifies the following choices:

| Model scale | Decoder choices |
|---|---|
| Nano / Small | HEB, dynamic upsampling, traditional head |
| Medium / Large / X-Large | HPB, dynamic upsampling, inverted head |

For the five FlexDepth variants, the width multipliers are \(0.25\), \(0.5\), \(1.0\), \(1.0\), and \(1.5\). The paper explicitly states that there is no per-pixel or per-region routing inside the decoder. The selection is static at the architecture level, while the upsampling itself is dynamic and content-aware through learned offsets \(\Delta \mathbf{P}\) [2607.00736].

## 3. Internal components

The post-convolution operator \(\mathcal{C}\) is instantiated as either the **High-Efficiency Bottleneck (HEB)** or the **High-Performance Bottleneck (HPB)**. HEB is used for Nano and Small models and is described as lightweight but favorable to gradient flow. Given an input feature \(\mathbf{X}\), it applies an initial convolution and channel split,
\[
[f_a, f_b] = \mathcal{S}(\mathrm{Conv}(\mathbf{X})),
\]
then a sequence of bottlenecks
\[
F_1 = B_1(f_a), \quad F_2 = B_2(F_1), \dots, F_n = B_n(F_{n-1}),
\]
followed by dense concatenation and a final convolution,
\[
\mathbf{X}_{out} = \mathrm{Conv}(\mathrm{Concat}[f_a, f_b, F_1, \dots, F_n]).
\]
The intended effect is dense connectivity and shortened gradient paths, which the paper identifies as critical for very small models such as Flex-Nano [2607.00736].

HPB is used for Medium, Large, and X-Large models. It replaces simple bottlenecks with CSP-based ones:
\[
B^k(x) = \mathrm{Conv}\Big(\mathrm{Concat}\big[ \prod_{j=1}^{n} B_j'(\mathrm{Conv}(x)), \ \mathrm{Conv}(x) \big]\Big).
\]
The stated effect is to split feature paths, process one branch through a stack of bottlenecks, concatenate it with a shortcut branch, and fuse the result with a final convolution. HPB is heavier than HEB but more expressive, and is enabled only where the added representational power is considered affordable [2607.00736].

The upsampling operator \(\mathcal{U}\) is a dynamic, content-aware module inspired by deformable sampling:
\[
\mathcal{U}(X)
 = \mathrm{GS}\!\left(X,\ \mathrm{PS}\!\left( \frac{2 \cdot (\mathbf{C} + \Delta \mathbf{P})}{\text{Norm} - 1}, s \right)\right),
\]
where \(X\) is the low-resolution feature map, \(\mathbf{C}\) is a base uniform sampling grid, \(\Delta \mathbf{P}\) is a learned offset predicted from \(X\), \(\text{Norm}\) is a coordinate normalizing factor, \(s\) is the upsampling factor, \(\mathrm{PS}\) is pixel shuffle, and \(\mathrm{GS}\) is grid-sample. The paper’s interpretation is that the module learns where to sample from the low-resolution feature map for each high-resolution location, which improves precision at object borders and thin structures relative to uniform bilinear interpolation [2607.00736].

The prediction head \(\mathcal{P}\) is also scale-dependent. Small models use a conventional head that predicts depth before upsampling. Medium and larger models use an inverted head that upsamples features first and predicts on high-resolution, high-channel features afterward. The stated rationale is that at large scale, predicting at high resolution improves local detail and depth discontinuities, while for small models the conventional head avoids extra full-resolution convolutions [2607.00736].

## 4. Supervision and static–dynamic decoupled training

SDD is used identically in both stages of FlexDepth training. In Stage 1, the pose network and the SDD-based depth network are trained jointly using standard self-supervised losses. The warping equation is
\[
\hat{I}_t = I_s \big\langle \mathrm{proj}(\hat{D}_t, P_{t \to s}, K) \big\rangle,
\]
the photometric error is
\[
L_p = \alpha \frac{1 - \text{SSIM}(\hat{I}_t, I_t)}{2} + (1-\alpha)\|\hat{I}_t - I_t\|_1,
\quad \alpha = 0.85,
\]
minimum reprojection with auto-masking is
\[
L_r = \mu \cdot \min_s L_p(\hat{I}_t, I_t),
\]
and edge-aware smoothness is
\[
L_{\text{smooth}} = |\partial_x d_t^*| e^{-|\partial_x I_t|} + |\partial_y d_t^*| e^{-|\partial_y I_t|}.
\]
All SDD depth predictions at scales \(s \in \{1, 1/2, 1/4, 1/8\}\) are supervised through
\[
L_{\text{raw}} = \frac{1}{4} \sum_{s \in \{1, 1/2, 1/4, 1/8\}} \big(L_r^{(s)} + \lambda L_{\text{smooth}}^{(s)}\big),
\quad \lambda = 0.001.
\]
This makes decoder quality directly visible to the self-supervised objective even though there is no SDD-specific regularizer [2607.00736].

Stage 2 introduces static–dynamic decoupling without changing the decoder architecture. Two checkpoints from Stage 1—an early checkpoint \(disp_i\) and a late checkpoint \(disp_l\)—are compared. The mask separating static and dynamic pixels is
\[
M = \big[\,|\text{disp}_i^\beta - \text{disp}_l^\beta| < g(\phi(f_{s_i}, f_{s_l})) \,\big],
\quad \beta = 0.7,
\]
where \(g(\phi(f_{s_i}, f_{s_l}))\) is a learned adaptive threshold. Pixels with stable predictions are labeled static; unstable regions are treated as dynamic [2607.00736].

In Stage 2, PoseNet is frozen and the depth network with the same SDD is retrained using a masked normal distribution loss,
\[
L_m = M \cdot \left[ \log(\sigma + 1) + \frac{\lambda L_r^2}{2\sigma^2} \right],
\quad \lambda = 0.5,
\]
a geometric depth smoothing term,
\[
L_{\text{GDS}} = |\partial_x \hat{d}_t| e^{-|\partial_x I_t|}
+ [\eta(1 - M) + M] |\partial_y \hat{d}_t| e^{-|\partial_y I_t|},
\quad \eta = 100,
\]
and the final loss
\[
L_{\text{total}} = \frac{1}{HW} \sum_t \big(L_m + \gamma L_{\text{GDS}}\big).
\]
A common misconception is that SDD itself performs dynamic-scene reasoning. The paper’s formulation is narrower: dynamic-scene handling comes from the two-stage training and mask-based losses, whereas the decoder is reused unchanged across both stages [2607.00736].

## 5. Efficiency, accuracy, and robustness

The FlexDepth results are reported on KITTI and Cityscapes, with SDD identified as central to the accuracy–efficiency trade-off. On KITTI at \(640 \times 192\), the five model variants are reported as follows:

| Model | Params / FLOPs | Abs Rel |
|---|---|---|
| Flex-Nano | 1.52M / 0.718G | 0.110 |
| Flex-Small | 6.06M / 2.78G | 0.104 |
| Flex-Medium | 12.72M / 9.99G | 0.096 |
| Flex-Large | 15.20M / 11.52G | 0.095 |
| Flex-X-Large | 32.27M / 24.56G | 0.093 |

On a Snapdragon 8 Elite at \(640 \times 192\) and batch size 1, the reported speeds are 37.6 FPS for Flex-Nano, 18.6 FPS for Flex-Small, 5.8 FPS for Flex-Medium, 5.2 FPS for Flex-Large, and 3.0 FPS for Flex-X-Large. The paper also states that Flex-Nano requires only 0.7 GFLOPs and achieves 37.6 FPS on mobile platforms [2607.00736].

The main benchmark claims attach specific SDD-enabled variants to specific accuracy points. On KITTI, Flex-Nano at 0.7G FLOPs matches or beats Lite-Mono-Tiny at similar AbsRel while using about one quarter of the FLOPs; Flex-Small at 2.8G outperforms Lite-Mono-Small and PuriLight; and Flex-X-Large at 24.6G achieves \(0.093\) AbsRel and \(0.910\) at \(\delta<1.25\). On Cityscapes at \(416 \times 128\), Flex-X-Large reaches AbsRel \(0.086\) and \(\delta<1.25 = 0.926\). The authors describe these results as state-of-the-art across arbitrary scales without auxiliary information and with minimal computational overhead [2607.00736].

Ablation studies isolate the contribution of the decoder. For X-Large on KITTI, HEB and HPB both yield \(0.093\) AbsRel, but HPB improves SqRel from \(0.614\) to \(0.605\) and RMSE from \(4.188\) to \(4.114\). Replacing dynamic upsampling with bilinear changes AbsRel from \(0.093\) to \(0.094\) while reducing FLOPs from \(24.6\)G to \(24.3\)G. Replacing the inverted head with a non-inverted head changes AbsRel from \(0.093\) to \(0.094\). Removing Stage 2 produces a larger degradation to \(0.100\) AbsRel, while a fixed-threshold mask recovers \(0.093\) and the adaptive dynamic mask achieves the best SqRel and RMSE at the same AbsRel [2607.00736].

The robustness claims are not limited to whole-image metrics. On Cityscapes dynamic regions, Flex-X-Large is reported to achieve dynamic-region AbsRel \(0.091\) and \(\delta<1.25 = 0.935\) with \(19.6\)G FLOPs. On KITTI moving-object analysis, FlexDepth improves AbsRel from \(0.177\) to \(0.159\) versus DSI-MonoViT, a \(10.2\%\) gain, and improves SqRel by \(22.6\%\). KITTI-trained Flex-X-Large also reaches the best Make3D metrics among lightweight baselines, with AbsRel \(0.279\) versus \(0.296\)–\(0.315\), and zero-shot evaluation on Cityscapes and DDAD is reported as stable [2607.00736].

## 6. Limitations, interpretation, and disambiguation

The paper explicitly limits the intended domain of SDD and FlexDepth to outdoor driving. Indoor environments with irregular textures, close-range clutter, and complex occlusions are not the focus, and performance there may be suboptimal. It also notes that the scale-driven mechanism is architecture-scale selection rather than per-image adaptive routing: there is no learned per-sample choice of HEB versus HPB or conventional versus inverted heads [2607.00736].

Several extensions are proposed in the source material. These include a Transformer-based SDD that integrates lightweight transformers into HPB while preserving dynamic upsampling; content-driven decoder routing that activates different decoder branches according to depth range or scene statistics; depth-range-aware heads for near, mid, and far ranges; multi-task extensions that share SDD with semantic segmentation or optical flow; and retuning for indoor or non-driving domains. These are presented as possible extensions rather than demonstrated properties [2607.00736].

The acronym **SDD** is also used elsewhere in the arXiv literature for unrelated concepts. In large language models, **Scale-Distribution Decoupling** denotes a re-parameterization of fully connected layers for stable Transformer training [2502.15499]. In knowledge distillation, **Scale Decoupled Distillation** denotes a logit distillation framework based on multi-scale local logits [2403.13512]. In anti-UAV detection, **SDD-YOLO** refers to a scale-driven detection design centered on a P2 high-resolution detection head [2603.25218]. In theoretical computer science, **SDD** commonly abbreviates **symmetric diagonally dominant** matrices and the associated parallel linear-system solvers [1311.3286; 1111.1750]. Within FlexDepth, however, SDD has the narrower and explicit meaning of a **Scale-Driven Decoder** for self-supervised monocular depth estimation in driving scenes [2607.00736].

Source: https://www.emergentmind.com/topics/scale-driven-decoder-sdd