GhostNetV2: Efficient CNN for Edge Devices

• The paper’s main contribution is a hardware-aware CNN architecture that reduces computation by decomposing feature maps using cheap operations.
• GhostNetV2 integrates CPU- and GPU-specific modules, optimizing inference speed on edge devices while maintaining high accuracy for classification and detection tasks.
• The framework leverages DFC attention to capture long-range dependencies, enhancing feature representation with efficient, low-cost operations for diverse vision applications.

GhostNetV2 is a family of light-weight convolutional neural network (CNN) architectures focused on overcoming the limitations of standard convolutions in resource-constrained environments. It systematically exploits feature map redundancy through “cheap operations,” introduces hardware-aware optimizations for CPUs and GPUs, and incorporates a hardware-friendly attention mechanism to capture long-range dependencies. The framework has been rigorously evaluated for image classification and object detection, demonstrating strong trade-offs between accuracy, model size, and speed, and has served as an effective drop-in backbone for efficient detection models on edge devices.

1. Principles of GhostNetV2 Design

GhostNetV2 builds on the foundational Ghost module, which reduces computational burden and parameter size by decomposing output feature map generation into two phases:

• Intrinsic Feature Computation: A reduced set of “intrinsic” feature maps $\mathbf{Y}'$ is derived from the input $\mathbf{X}$ using standard convolution with $m$ filters, with $m \ll n$ (the desired output channel count):

$\mathbf{Y}' = \mathbf{X} * \mathbf{f}'$

• Ghost Map Generation: Each intrinsic feature $y_i'$ serves as the input to $s$ inexpensive linear transformations $\Phi_{i,j}$ (e.g., depthwise convolutions), yielding a total of $n = m \cdot s$ output maps:

$y_{i,j} = \Phi_{i,j}(y_i'), \quad j = 1, \ldots, s$

The last transformation is usually the identity, preserving the original intrinsic feature. All ghost maps are concatenated to form the final output.

These strategies are extended in GhostNetV2 to specifically address hardware diversity and the need to capture long-range dependencies. Two variants, C-Ghost and G-Ghost, target CPU and GPU optimization respectively (Han et al., 2022), and an efficient attention module (DFC attention) expands the receptive field without incurring significant computational cost (Tang et al., 2022).

2. Architectural Innovations: C-Ghost and G-Ghost

GhostNetV2 introduces CPU- and GPU-optimized sub-architectures:

Variant	Optimization Target	Core Module	Description
C-GhostNet	CPU	C-Ghost module, C-Ghost bottleneck	Uses two stacked C-Ghost modules per block, favoring 1×1 pointwise convolutions and minimal depthwise operations.
G-GhostNet	GPU	G-Ghost stage structure	Exploits stage-wise redundancy: intrinsic features computed with thin multi-layer blocks; ghost features generated from the stage input using GPU-friendly ops.

C-Ghost Module and Bottleneck

• Bottlenecks consist of two C-Ghost modules: the first for channel expansion, the second for reduction, connected via shortcuts. In downsampling (stride 2), an extra depthwise convolution and downsampled shortcut are applied.
• The “stem–body–head” network pattern incorporates overall efficiency, with width multiplier $\alpha$ providing scalable control over resource consumption.

G-Ghost Stage and Aggregation

• The G-Ghost stage divides output into “complicated” (intrinsic) features, computed sequentially by thin blocks, and “ghost” features, computed in parallel by a single cheap operation on the stage input.
• Feature aggregation (mix operation) leverages global pooling and a small fully connected layer to supplement ghost features with intermediate information.

These innovations minimize the number of heavy operators, reduce memory movement, and take device-specific operator efficiency into account.

3. Hardware-Friendly Attention: DFC Attention

GhostNetV2 overcomes the receptive field bottleneck of cheap operations by integrating Decoupled Fully Connected (DFC) attention (Tang et al., 2022), designed to be deployable on edge hardware:

• Decoupling Full-Space Aggregation: A full $(HW) \times (HW)$ spatial aggregation is split into two sequential 1D processes:
  • First, aggregation along rows by a $1 \times K_H$ depthwise convolution.
  • Second, aggregation along columns by a $K_W \times 1$ depthwise convolution.
• Reduction in Complexity: Complexity drops from $O((HW)^2)$ to $O(K_H H W + K_W H W)$. These operations are implementable with standard deep learning frameworks and avoid reshaping overhead.
• Attention Fusion: DFC attention outputs are used to reweight local features via element-wise multiplication after upsampling to the original spatial size.

A typical GhostNetV2 block integrates both the traditional Ghost branch and the DFC attention branch, fusing their outputs as follows:

def GhostNetV2_Block(X): # Ghost branch Y_intri = Conv1x1(X) Y_exp = DepthwiseConv(Y_intri) Y = Concat(Y_intri, Y_exp) # DFC attention branch Z = Conv1x1(X) Z_down = Downsample(Z) Z_h = DepthwiseConv_horizontal(Z_down) Z_v = DepthwiseConv_vertical(Z_h) A = Sigmoid(Z_v) A_up = Upsample(A) # Fuse (reweight) Output = A_up * Y return Output

Downsampling and upsampling around the attention block maintain efficiency. DFC attention can efficiently enhance long-range interactions, which is critical for accurate recognition in compact models.

4. Performance Evaluation and Use Cases

GhostNetV2 exhibits notable improvements over GhostNetV1 and other compact architectures in both accuracy and efficiency:

• Classification: On ImageNet, GhostNetV2 (1.0×) achieves 75.3% top-1 accuracy with 167M FLOPs, surpassing GhostNetV1 1.1× (74.5%) at similar computational cost. Practical CPU inference latencies are on par with baseline GhostNet (≈37–38 ms on Kirin 980) (Tang et al., 2022).
• Object Detection: As a YOLO backbone, GhostNetV2 supports significant reductions in parameter count and FLOPs. When used for safety helmet detection, a GhostNetV2-based YOLO model (with attention modules and the GAM optimizer) achieves a 2% mAP increase and reduces parameter count by more than 25% (Shen et al., 4 May 2024).
• Generalization and Edge Computing: Integration with attention modules (e.g., SCNet and Coordinate Attention) allows the model to preserve or enhance spatial expressiveness, even after aggressively reducing parameters. The inclusion of optimization strategies such as GAM further improves generalization by encouraging flatter minima in the loss landscape.

Typical applications include mobile vision (real-time recognition on smartphones), embedded object detectors (drones, industrial safety analytics), and IoT devices requiring low latency and power usage.

5. Training and Optimization Strategies

GhostNetV2’s performance can be further enhanced by advanced training strategies, as demonstrated in subsequent work (Liu et al., 17 Apr 2024):

• Re-parameterization: Additional parallel branches (e.g., adding 1×1 depthwise convolutions in parallel during training) provide richer feature learning but are “folded” into a single operator for inference. The combined weights and biases are:

$$W_{\text{rep}} = \sum_i \hat{W}_i \qquad b_{\text{rep}} = \sum_i \hat{b}_i$$

• Knowledge Distillation (KD): Training with a teacher network, using a blended loss function:

$$\mathcal{L}_{\text{total}} = (1-\alpha) \mathcal{L}_{ce}(\Gamma_s(x), y) + \alpha \mathcal{L}_{kd}(\Gamma_s(x), \Gamma_t(x))$$

where $\mathcal{L}_{kd}$ is typically a temperature-scaled KL divergence.

• Augmentation and Scheduling: The choice of data augmentation is crucial; operations like Mixup and CutMix may harm compact model performance, contrasting with their benefits for standard models.

Deployment of these techniques results in additional gains: e.g., GhostNetV2 1.3× (retrained with these strategies) increases top-1 ImageNet accuracy from around 76.9% to 79.1% with stable latency and computational cost, indicating a substantial accuracy–efficiency improvement.

6. Integrations and Application Case Studies

GhostNetV2 functions as a plug-in module for existing detection architectures, benefiting tasks such as helmet detection (Shen et al., 4 May 2024):

• YOLO Integration: Substitution of standard C3 and convolution modules with GhostNetV2’s GhostConv and GhostC3. The GhostNet block’s two-phase feature generation offers ≈25% parameter reduction with no loss in feature richness, facilitated by the following process for each input $X$:

$Y' = X * F_{1 \times 1}$

$Y = \text{Concat}(Y', Y' * F_{dp})$

• Attention Module Augmentation: SCNet is integrated for multi-scale spatial context capture. Coordinate Attention (CA) propagates spatial location information by pooling along height and width, concatenating and transforming via 1×1 convolutions.
• Optimization via GAM: The GAM optimizer, by penalizing sharpness (largest Hessian eigenvalue $\lambda_{\text{max}}$ and first-order flatness $R_p^{(1)}$), improves generalization, accelerating convergence to flat minima.

Ablation studies confirm that this combination leads to improved mAP, more compact parameter footprint, and more robust generalization in real-world detection scenarios.

7. Prospects and Research Directions

The architectural principles of GhostNetV2 substantiate several open directions:

• Automated Design: Neural architecture search tailored to hardware constraints may yield further optimized configurations.
• Attention Mechanism Extensions: Exploring alternate lightweight attention mechanisms and dynamic kernel arrangements may improve expressiveness while preserving efficiency.
• Task Generalization: Adaptation of these concepts to semantic segmentation, video analysis, and other domains is plausible and may require only minor adjustment of integration points.
• Hybridization: Merging hardware-friendly attention (DFC) with transformer-style modules or other expressiveness boosters could unlock additional accuracy gains as on-device hardware evolves.

A plausible implication is that the principles underlying GhostNetV2 will continue to serve as the foundation for efficient neural architecture design across an expanding range of edge applications.