Grad-CAM++: Refined CNN Visual Explanations

• Grad-CAM++ is a gradient-based technique that refines CNN explanations by incorporating pixel-wise and positive gradient weighting for sharper heatmaps.
• It enhances object localization by leveraging higher-order derivative information, yielding more complete coverage even for multiple instances.
• Empirical evaluations show improved mIoU and increased confidence scores on benchmarks like ImageNet and PASCAL VOC compared to the original Grad-CAM.

Grad-CAM++ is a gradient-based class activation mapping technique designed for the visual explanation of convolutional neural network (CNN) predictions. It addresses limitations inherent in the original Grad-CAM method by introducing pixel-wise weighting of gradients, enabling improved object localization and more effective handling of multiple instances of the same class within an image. Grad-CAM++ achieves this by leveraging higher-order partial derivatives and spatial weighting within feature maps, producing class-discriminative heatmaps that are both sharper and more comprehensive than those generated with previous approaches.

1. Motivation and Context

The foundational Grad-CAM approach generates visual explanations by globally averaging the gradients of the class-specific score with respect to the feature maps from a convolutional layer. These global weights are then applied to the corresponding feature maps, after which a ReLU operation creates the final class-activation map. However, this method often highlights only the most discriminative region of an object, resulting in incomplete coverage, and struggles to adequately capture multiple instances of the same class. Grad-CAM++ was introduced specifically to remedy these deficiencies via pixel- and location-dependent weighting of the gradient information, enabling both richer object coverage and accurate multi-instance representation (Chattopadhyay et al., 2017).

2. Mathematical Formulation

Given a convolutional layer with $k$-th feature map $A^k\in\mathbb{R}^{H\times W}$, and a target class score $Y^c$ (commonly the pre-softmax or exponential of pre-softmax), the standard Grad-CAM weight is: $$w_k^c = \frac{1}{Z} \sum_{i,j} \frac{\partial Y^c}{\partial A^k_{ij}}\ , \quad Z = H \cdot W.$$ The output heatmap is then: $L^c_{\rm Grad\mathchar`-CAM}(i, j) = \mathrm{ReLU}\Bigl(\sum_k w_k^c\,A^k_{ij}\Bigr).$ Grad-CAM++ modifies this by employing spatially varying, data-driven pixel weights $\alpha_{ij}^{k, c}$: $$w_k^c = \sum_{i,j} \alpha_{ij}^{k, c} \cdot \mathrm{ReLU}\left( \frac{\partial Y^c}{\partial A^k_{ij}} \right).$$ The weight coefficients $\alpha_{ij}^{k,c}$ are determined such that the reconstruction

$Y^c = \sum_k w_k^c \sum_{i,j} A^k_{ij}$

holds. Through differentiation and under the commonly used $Y^c = \exp(S^c)$, the weights take the closed form

$A^k\in\mathbb{R}^{H\times W}$0

This formula ensures that both the magnitude and spatial context of each gradient contribute to the final heatmap (Chattopadhyay et al., 2017).

3. Algorithmic Implementation

The Grad-CAM++ pipeline consists of the following steps (Chattopadhyay et al., 2017):

1. Forward pass: Run the input through the network, cache activations $A^k\in\mathbb{R}^{H\times W}$1, and compute class score $A^k\in\mathbb{R}^{H\times W}$2 (or penultimate $A^k\in\mathbb{R}^{H\times W}$3).
2. Backward pass: Compute gradients $A^k\in\mathbb{R}^{H\times W}$4 for the chosen layer.
3. Compute higher-order terms: Obtain squares and cubes of the gradients and spatial sums of feature maps.
4. Evaluate $A^k\in\mathbb{R}^{H\times W}$5: For each channel/location, use the closed-form as above.
5. Aggregate weighted gradients: Calculate $A^k\in\mathbb{R}^{H\times W}$6 by summing over weighted, thresholded (positive) gradients.
6. Generate the heatmap: Apply the weighted sum to activations, apply ReLU, and spatially upsample to input resolution.

Optionally, the final map can be fused with a Guided Backpropagation mask to enhance resolution. Only positive gradients are propagated to capture feature importance increasing the class score.

4. Empirical Evaluation and Observed Advantages

Grad-CAM++ has been validated on standard benchmarks such as ImageNet and PASCAL VOC. On ImageNet validation (VGG-16), Grad-CAM++ achieves a lower Average Drop percentage (36.8% vs 46.6%) and higher Increase in Confidence (17.1 vs 13.4) relative to Grad-CAM. Multi-label Pascal VOC comparisons also show better object coverage, with notable improvements in mIoU for weakly-supervised localization (0.38 vs 0.28) (Chattopadhyay et al., 2017).

Qualitative analysis indicates that Grad-CAM++ produces sharper and more complete localization, properly highlighting multiple object instances, whereas Grad-CAM tends to ignore smaller or less salient instances and focuses disproportionately on the most discriminative region.

5. Equivalence to Positive-Gradient Grad-CAM

Subsequent analysis demonstrates that the practical improvement of Grad-CAM++ over Grad-CAM stems mainly from the use of positive gradients, rather than the fine-grained higher-order weighting (Lerma et al., 2022). Empirically, the per-pixel coefficients $A^k\in\mathbb{R}^{H\times W}$7 are nearly constant (clustered tightly around $A^k\in\mathbb{R}^{H\times W}$8 for nonzero gradients) across diverse images and architectures. Thus, the Grad-CAM++ weights can be closely approximated by simply averaging the ReLU-thresholded gradients: $A^k\in\mathbb{R}^{H\times W}$9 Consequently, the class-discriminative map produced by Grad-CAM++ is practically equivalent to a Grad-CAM variant using only positive gradients. Performance comparisons exhibit near-identical results, both quantitatively and qualitatively, confirming that the empirical benefits are due to discarding negative gradients rather than to the complex higher-order $Y^c$0 weights.

6. Extensions and Generalizations

The Grad-CAM++ alpha-weighting formulation serves as the foundation for further developments. Smooth Grad-CAM++ incorporates noise-averaging (from SmoothGrad) by injecting Gaussian perturbations at inference time and averaging gradient and higher-order responses before $Y^c$1 computation, yielding maps with both enhanced sharpness and improved multi-instance separation. On the PASCAL VOC object localization task, Smooth Grad-CAM++ improves mIoU to 0.52 relative to 0.46 for Grad-CAM++ and 0.40 for Grad-CAM (Omeiza et al., 2019).

Recent generalizations such as Integrative CAM adopt a broader alpha-term formulation that applies to any smooth output function, incorporates classifier bias, and adaptively fuses information from multiple layers for improved interpretability in complex CNNs (Singh et al., 2024). However, these methods remain rooted in the spatial weighting and gradient-based explanation framework innovated by Grad-CAM++.

7. Recommendations and Practical Implications

For practitioners, the complexity of computing third-order derivatives and precise $Y^c$2 coefficients in Grad-CAM++ can typically be eschewed in favor of the ReLU-thresholded gradient-based variant, which yields equivalent localization performance at much lower computational and numerical cost (Lerma et al., 2022). For most applications, using positive gradients—effectively employing Grad-CAM$Y^c$3 as an Editor's term—is sufficient to capture all practical advantages previously attributed to Grad-CAM++. Care must be taken to apply the ReLU before any spatial averaging or pooling of gradients to realize these benefits.

In summary, Grad-CAM++ advances the explainability of convolutional neural networks by leveraging spatially detailed, positive-gradient-weighted class activation mapping, and its principles underpin a spectrum of subsequent techniques in explainable deep learning.