---
title: ConvNeXt-tiny RGB Branch Overview
url: https://www.emergentmind.com/topics/rgb-branch-convnext-tiny
type: topic
---

# ConvNeXt-tiny RGB Branch Overview

An RGB branch based on ConvNeXt-tiny represents a state-of-the-art convolutional backbone for extracting global visual features from RGB images. This architectural component appears across diverse tasks such as deepfake detection, manipulation localization, image classification, 3D object detection, and image dehazing. It leverages ConvNeXt’s hierarchical spatial feature extraction, depthwise separable convolutions, GELU activations, and channel-wise LayerNorm. Variations in input resolution, branching, attention fusion, loss supervision, and training regimes adapt this core design to the specific requirements of each application. The following sections synthesize the technical characteristics, integration paradigms, and empirical justifications for the RGB branch with ConvNeXt-tiny backbone.

## 1. Core Architecture and Computational Blocks

All reviewed works employ the ConvNeXt-tiny framework, which decomposes the image into progressively abstract spatial features through a patch embedding stem and four successive stages of inverted-bottleneck blocks. The standard configuration comprises 3, 3, 9, 3 residual blocks in stages 1–4, with channel widths 96, 192, 384, 768. Each block combines:

- Channel-wise LayerNorm.
- Depthwise 7×7 (or 3×3) convolution (groups = input channels).
- Pointwise 1×1 convolution for channel expansion (expansion ratio 4), followed by GELU, then 1×1 projection back to base width.
- Residual addition.

The pseudocode signature for a canonical ConvNeXt block is:
```python
x₀ = LayerNorm(x)
x₁ = DepthwiseConv7×7(groups=C)(x₀)
x₂ = Conv1×1(C→4C)(x₁)
x₃ = GELU(x₂)
x₄ = Conv1×1(4C→C)(x₃)
y = x + x₄
```
Some works reduce depth/width for efficiency: for example, EVCC employs depths [2,2,6,2], widths [64,128,256,512], yielding lower computational cost while preserving the design philosophy [2511.18691].

## 2. Input Processing and Data Augmentation

The RGB branch’s input protocol typically includes:

- Precise spatial preprocessing: Face alignment and center cropping to 224×224 (ForensicFlow [2511.14554]), resizing to 256×256 (Noise and Edge dual-branch [2409.00896]), or, for object detection, 480×480 (MonoNext [2308.00596]).
- Channel-wise normalization using ImageNet mean and standard deviation statistics.
- Training augmentations including random horizontal flipping (p=0.5), brightness and contrast jitter, minor geometric transforms (rotation ±5°, resized crops 0.9–1.0), and task-specific MixUp (EVCC) [2511.18691].
- Input tensor shape post-augmentation is either B×3×224×224 or B×3×H×W depending on application.

## 3. Output, Embedding, and Decoder Stages

After the final ConvNeXt stage, the feature map is post-processed to yield task-specific embeddings:

| Downstream Task              | Output After ConvNeXt    | Projection/Pooling          | Final Shape           |
|------------------------------|--------------------------|-----------------------------|-----------------------|
| Video Deepfake Detection     | 7×7×768                  | GAP → Linear (768→512)      | ℝ⁵¹²                  |
| Manipulation Localization    | 8×8×768                  | Upsample+FeatureEnh. Module | 8×8×768 → pixel mask  |
| Image Classification         | 14×14×512                | Flatten → Linear (512→384)  | ℝ³⁸⁴                  |
| 3D Object Detection (BEV)    | 15×15×128*               | Head-specific convolutions  | 15×15×(task channels) |
| Non-homogeneous Dehazing     | Variable†                | U-Net style upsampling      | H×W×3                 |

\* After four ConvNeXt-style blocks post-MobileNetV2 backbone; † after upsampling via three PixelShuffle+attention blocks.

Global average pooling is standard for video and classification tasks; spatial upsampling/decoding branches dominate in pixel-level localization or regression settings [2511.14554, 2409.00896, 2511.18691, 2308.00596, 2305.04430].

## 4. Fusion Strategies and Attention Mechanisms

Fusion paradigms adapt the RGB branch output for multi-branch architectures:

- **Temporal Attention**: ForensicFlow generates per-frame embeddings, then aggregates K-frame segments using attention-driven pooling:
  $$
  F^{RGB}_{video} = \sum_{t=1}^K \alpha_t \cdot f_t^{RGB}, \quad \alpha_t = \frac{\exp(e_t)}{\sum_{j=1}^K \exp(e_j)}
  $$
  with $e_t$ from an MLP + softmax [2511.14554].
- **Multi-Modal/Branch Fusion**: Adaptive attention weighting for late fusion, as in ForensicFlow and EVCC:
  $$
  F_{fused} = \gamma_{RGB} F_{RGB} + \gamma_{Tex} F_{Tex} + \gamma_{Freq} F_{Freq}
  $$
  with $\gamma$ produced by a learnable softmax or, in EVCC, by a fusion router with confidence modulation [2511.14554, 2511.18691].
- **Cross-Attention**: EVCC applies gated, bidirectional cross-attention between ConvNeXt and Transformer tokens after token pruning, refining semantic alignment [2511.18691].
- **Spatial Fusion**: Manipulation localization and dehazing architectures sum or blend branch outputs per-pixel, optionally after applying attention or learned blending maps [2409.00896, 2305.04430].

## 5. Loss Functions and Optimization

Each network configures loss and training schedules to suit the detection, localization, or generative target:

- **Global/Binary Tasks (e.g., Deepfake Detection)**: Focal Loss with $α=1.0, γ=2.0$ for severe class imbalance [2511.14554].
- **Segmentation/Localization**: Joint supervision with BCE, Focal, and Edge Dice losses targeting both per-pixel masks and boundary localization:
  $$
  \mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{BCE}} + \mathcal{L}_{\mathrm{FL}} + \mathcal{L}_{\mathrm{edge}}
  $$
  where the edge loss explicitly sharpens structure awareness [2409.00896].
- **Regression/Image Restoration**: Per-branch L1 and perceptual (VGG) loss, MS-SSIM for structure, and adversarial GAN losses for photo quality (in dehazing) [2305.04430].
- **Detection**: Sum of confidence, class, box (IoU and angle) losses, with explicit loss weights for each aspect as in 3D object detection [2308.00596].
- Common optimization settings include AdamW/Adam, learning rates (10⁻⁴, 2×10⁻⁵), learning rate decay (per-unfreeze or schedule), and progressive unfreezing for transfer learning [2511.14554, 2511.18691].

## 6. Architectural Adaptations and Contextual Significance

Task-dependent modifications include:

- **Downsampling Variations**: Input size and patch/stem stride redefine downsampled feature map dimensions (e.g., 64×64@96 for 256×256 input [2409.00896]; 7×7@768 for 224×224 [2511.14554]).
- **Stage Truncation and Lightweighting**: For efficiency, some branches omit final stages (e.g., Dehazing omits Stage 4 [2305.04430]) or shrink width/depth factors (EVCC) [2511.18691].
- **Branch-Specific Heads**: Detection/Restoration tasks append task-tuned decoders over the ConvNeXt output, while classification or video understanding ultimately reduce to global embeddings [2511.14554, 2308.00596].
- **Token Pruning and Late Routing**: In hybrid networks such as EVCC, adaptive token pruning halves the length of the ConvNeXt output sequence and fuses it with other modalities via dynamic gates, reducing FLOPs by 25–35% while maintaining accuracy [2511.18691].
- **Attention and Feature Enhancement**: Modules such as Feature Enhancement (F_enh), per-pixel blending weights, and multi-scale channel attention refine spatial detail prior to fusion or decoding [2409.00896, 2305.04430].

## 7. Summary Table: RGB Branch (ConvNeXt-tiny) Design Across Representative Systems

| System (arXiv)     | Input Size | Depths (Blocks)     | Widths            | Output Embedding         | Key Fusion/Decoder     | Main Losses         |
|--------------------|------------|---------------------|-------------------|-------------------------|-----------------------|---------------------|
| ForensicFlow [2511.14554]    | 224×224    | [3,3,9,3]           | [96,192,384,768]  | 512-dim (per frame)     | Attention pooling, tri-modal fusion | Focal (α,γ)         |
| Noise+Edge [2409.00896]      | 256×256    | [3,3,9,3]           | [96,192,384,768]  | 8×8×768                 | FE module, pixel-wise fusion        | BCE, Focal, Edge    |
| EVCC [2511.18691]            | 224×224    | [2,2,6,2]           | [64,128,256,512]  | 384-dim                 | Pruning, cross-attention, router    | Cross-entropy       |
| MonoNext [2308.00596]        | 480×480    | 4 ConvNeXt blocks*   | [512,256,256,128] | 15×15×128 (BEV grid)    | 5 heads: class/box/yaw/score        | L2, IoU, angle      |
| Dehazing [2305.04430]        | H×W        | [3,3,9]              | [96,192,384]      | Variable spatial maps   | U-Net upsampling, per-pixel blend   | L1, perceptual, MS-SSIM |

\* After MobileNetV2 backbone.

## References

- ForensicFlow [2511.14554]
- Noise and Edge dual-branch [2409.00896]
- EVCC [2511.18691]
- MonoNext [2308.00596]
- Breaking Through the Haze [2305.04430]

The ConvNeXt-tiny-based RGB branch establishes a robust, extensible foundation for integrating global spatial semantic information in vision models, and its variants, by virtue of carefully balanced architectural hyperparameters and task-adaptive fusion strategies, demonstrate strong empirical performance across detection, localization, recognition, and restoration modalities.

Source: https://www.emergentmind.com/topics/rgb-branch-convnext-tiny