Decom-CAM: Fine-Grained Neural Interpretability

• The paper presents a two-stage framework that decomposes class activations into orthogonal feature maps and integrates them via measured causal contributions.
• Decom-CAM leverages singular value decomposition to uncover distinct semantic components, offering more granular and diagnostically useful visualizations than standard CAM techniques.
• Empirical evaluations show improved localization and accuracy metrics, demonstrating Decom-CAM’s computational efficiency and enhanced interpretability in deep models.

Decom-CAM is a two-stage deep neural network interpretability method that produces feature-level, fine-grained saliency maps by decomposing class-discriminative activation maps into orthogonal sub-features and integrating them via their measured causal contributions to decision scores. Developed to address the entanglement and lack of granularity in standard Class Activation Map (CAM) approaches, Decom-CAM and its successors (notably DecomCAM for vision-LLMs) leverage singular value decomposition (SVD) to expose the semantic structure underlying CNN decision-making, overcoming the limitations of prior interpretability tools while maintaining computational efficiency (Yang et al., 2023, Yang et al., 2024).

1. Motivations and Conceptual Foundations

Existing CAM-based methods—such as CAM, Grad-CAM, and ScoreCAM—generate class-specific saliency maps by aggregating convolutional activations linearly with pre-computed or gradient-based channel weights. While these heatmaps highlight spatial regions relevant for a model’s classification, they do not unravel how the model internally separates semantic parts (e.g., heads, eyes, wings, artifacts) or mitigate noise from redundant or background channels. Such limitations are aggravated in modern architectures (e.g., CLIP), where overparametrized last-stage convolutional representations encode numerous entangled and irrelevant features (Yang et al., 2024). Additionally, the reliance on global channel weighting obscures the contributions of distinct object parts, collapsing all relevant evidence into a single coarse map.

Decom-CAM was introduced as a solution to this bottleneck by:

• Decomposing class-discriminative activations into a small number of orthogonal (mutually uncorrelated) components—each capturing a distinct semantic sub-feature.
• Integrating these orthogonal maps according to their quantitative causal contribution to the model’s class score. This decomposition-plus-integration paradigm provides a more granular and diagnostic interpretability than traditional channel-wise aggregation (Yang et al., 2023, Yang et al., 2024).

2. Mathematical Formulation and Saliency Computation

The Decom-CAM workflow operates as follows:

A. SVD-Based Decomposition of Class-Discriminative Activations

1. Given a CNN and input image $I\in\mathbb{R}^{c_0\times H\times W}$, extract the activation tensor $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$ at layer $k$.
2. Compute the class-discriminative activation map

$$S_k = \frac{\partial y_\ell}{\partial A_k} \odot A_k \in \mathbb{R}^{c_k\times m\times n},$$

where $y_\ell$ is the network’s class-logit of interest and $\odot$ denotes the elementwise (Hadamard) product.

1. Flatten spatial dimensions to form

$$S_k^* = [s_{1}, s_{2}, ..., s_{c_k}] \in \mathbb{R}^{c_k\times (mn)},$$

with $s_i\in\mathbb{R}^{mn}$ for each channel.

2. Apply singular value decomposition:

$S_k^* = U_k\Sigma_k V_k^T.$

3. Retain the top $p$ singular vectors:

$A_k(I)\in\mathbb{R}^{c_k\times m\times n}$0

4. Project $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$1 onto the $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$2 principal directions:

$A_k(I)\in\mathbb{R}^{c_k\times m\times n}$3

5. Reshape each row to obtain $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$4 orthogonal feature maps (OFMs)

$A_k(I)\in\mathbb{R}^{c_k\times m\times n}$5

where each $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$6 isolates a distinct principal component.

B. Causal Integration and Saliency Map Synthesis

6. Upsample each feature map to input size and scale to $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$7:

$A_k(I)\in\mathbb{R}^{c_k\times m\times n}$8

1. For each $A_k(I)\in\mathbb{R}^{c_k\times m\times n}$9, generate a masked input $k$0, where $k$1 is Gaussian-blurred $k$2.
2. Compute the causal contribution of each OFM:

$k$3

with $k$4 as the class-specific model output.

1. Apply softmax normalization:

$k$5

2. Integrate into the final heatmap:

$k$6

This process enables Decom-CAM to localize object parts and disentangle disjoint semantic features, resulting in more faithful and interpretable saliency at minimal computational cost (Yang et al., 2023, Yang et al., 2024).

3. Comparison with Prior CAM Variants

Method	Spatial Decomposition	Local Semantics	Multiple Features	Weighting Mechanism
CAM	None	No	No	Learned linear layer
Grad-CAM	None	No	No	Channel gradient weights
Grad-CAM++	None	No	No	Refined gradients
ScoreCAM	No	No	No	Score drop per channel
EigenCAM	PCA	Limited	One principal	Largest PCA component
Decom-CAM	SVD-based	Yes	Yes	Causal contribution

Conventional CAM and Grad-CAM aggregate feature maps linearly (with fixed or gradient-based weights), outputting a single, often coarse, saliency map. Recent variants such as ScoreCAM and EigenCAM attempt to reweight or project activations for improved interpretability, but still suffer from entanglement and lack of explicit semantic partitioning. Decom-CAM’s decomposition step explicitly generates multiple orthogonal submaps, each interpretable as a unique semantic component (e.g., "eye," "tail," "whiskers"), allowing nuanced visualization of model reasoning previously obscured by coarse averaging (Yang et al., 2023, Yang et al., 2024).

4. Evaluation Protocol and Empirical Performance

A rigorous evaluation protocol was proposed for Decom-CAM to disentangle interpretability from model errors:

• The test set is partitioned into bins by classification accuracy (width 10%); only correctly predicted samples are evaluated within each bin.
• Deletion and insertion metrics are measured: for deletion, pixels are removed in descending order of saliency (AUC over score drop, lower is better); for insertion, pixels are revealed in order of increasing saliency (AUC over score recovery, higher is better). The overall score is computed as insertion AUC minus deletion AUC.
• The Pointing Game metric on PASCAL VOC calculates accuracy as the fraction of maximum-saliency points falling within a ground-truth bounding box.

Empirical results:

• Pointing Game (VOC 2012, ResNet-50 CLIP backend):
  • Grad-CAM++: 85.70%
  • Decom-CAM: 86.86%
• Deletion/Insertion scores (overall accuracy bins):

| Method | Insertion (%) ↑ | Deletion (%) ↓ | Overall (%) ↑ | |--------------|----------------|---------------|---------------| | Grad-CAM | 47.56 | 12.80 | 34.76 | | Grad-CAM++ | 48.18 | 11.71 | 36.47 | | Decom-CAM | 52.67 | 12.87 | 39.80 |

DecomCAM (the generalized variant for VLMs and multi-object scenes) mirrors these trends on multiple benchmarks and backbones (CLIP, RN50×4, RN50×16), delivering higher MaxBoxAccV2 and pointing game accuracy. DecomCAM attains a runtime of ≈0.182 s per inference on CLIP-RN50, compared to ScoreCAM's 45 s and GradCAM's 0.148 s, achieving a favorable balance of interpretability and efficiency (Yang et al., 2023, Yang et al., 2024).

5. Interpretability Insights and Granular Analysis

Decom-CAM's decomposition exposes explicit object parts and contextually distinct features:

• Each orthogonal feature map typically corresponds to a spatially and semantically distinct region—e.g., separate OFMs localize "eyes," "nose," "body," or "wheels."
• In vision-LLMs, DecomCAM demonstrates similar part-level disentanglement for textual prompts (e.g., splitting "dog" and "shirt" in "dog wearing shirt," or "person" from "chair").
• Attribute-level analyses on datasets such as Pascal-Part further reveal that the top principal components (first few OFMs/OSSMs) typically account for the majority of the salient object regions used in model predictions. Empirically, the first OSSM can achieve near 100% pointing game recall for several categories, while subsequent components offer diminishing marginal contribution.

A plausible implication is that a small set of orthogonal maps can capture the most discriminative visual cues exploited by large-scale deep models, mitigating redundancy and clarifying the model’s internal representation (Yang et al., 2023, Yang et al., 2024).

6. Implementation and Computational Complexity

• Backbone and Layer Choices: Decom-CAM is implemented on the final convolutional stage of CLIP image encoders (e.g., RN50, RN50×4, RN50×16), typically operating on hundreds to a few thousand channels.
• Hyperparameters: Common choices are $k$7 channels and $k$8 OSSMs for default multi-object interpretability; higher values are used for single-object or attribute-part datasets.
• Computational Cost: SVD is performed on a $k$9 matrix (with MN the spatial resolution), which is efficient for practical $$S_k = \frac{\partial y_\ell}{\partial A_k} \odot A_k \in \mathbb{R}^{c_k\times m\times n},$$0. Integration requires only $$S_k = \frac{\partial y_\ell}{\partial A_k} \odot A_k \in \mathbb{R}^{c_k\times m\times n},$$1 forward passes per sample. Memory overhead is primarily from storing decomposition matrices and OSSMs.
• Extension to Vision-LLMs: For VLMs, the target score can be the cosine similarity between image and textual label embeddings.

This ensures that Decom-CAM and DecomCAM are scalable and applicable to contemporary large-scale models without major inference overhead (Yang et al., 2024).

7. Major Contributions, Limitations, and Future Directions

Key contributions include:

• Formulation of a two-stage interpretability pipeline: first decomposing class-discriminative activations into orthogonal feature maps, then re-integrating them by causal score contribution.
• Empirical demonstration of improved accuracy, sharper localization, and richer semantic attribution compared to existing CAM-based techniques.
• Introduction of a rigorous, accuracy-stratified evaluation protocol decoupling interpretability from model errors.

Potential future research directions encompass:

• Extension to multi-layer or multi-class decompositions for richer cross-layer semantic tracing.
• Exploration of alternative matrix decompositions (e.g., nonnegative matrix factorization) for increased spatial localization.
• Human-in-the-loop or interactive interpretability, especially for diagnostic or trustworthiness assessment scenarios.
• Scalability and generalization to very large vision architectures (e.g., vision transformers) and verification of cross-architecture robustness (Yang et al., 2023, Yang et al., 2024).

Decom-CAM and its successors present a paradigm shift in feature-level interpretation, offering a diagnostic window into deep models that is both granular and computationally efficient.