Importance Annealing for CNN Interpretability

• Importance Annealing is an explainable-AI method that selects the most relevant CNN feature maps using quantum annealing for post-hoc interpretation.
• The method formulates feature selection as a QUBO problem, balancing Grad-CAM-derived importance scores against pairwise redundancy with a tunable trade-off parameter.
• Empirical evaluations on STL-10 with a ResNet-18 showed that the quantum annealing approach yields disentangled, class-specific explanations with competitive average drop percentages.

Importance Annealing is an explainable-AI method for post-hoc interpretation of convolutional neural networks that formulates feature-map selection as a Quadratic Unconstrained Binary Optimization problem and solves it with quantum annealing. Its central objective is to identify, for a given input image and predicted class, a subset of final-layer feature maps that is simultaneously high-importance and low-redundancy. In this formulation, interpretability is not treated as a heat-map construction problem alone, but as a combinatorial selection problem over learned representations, with the selected subset used to produce disentangled explanations of class-specific reasoning (Venturelli et al., 28 Apr 2026).

1. Problem formulation and definition of “importance”

In Importance Annealing, an input image $x$ is passed through a pretrained CNN whose final convolutional layer yields feature maps

$$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$

and a scalar logit $z=\mathrm{CNN}(x)$ for the predicted class. The method defines the importance of feature map $a$ using the Grad-CAM-style global average of its output gradient,

$$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$

Positive $\alpha_a$ is interpreted as class-confirming, whereas negative values are class-opposing. The procedure therefore filters to the $d\le N_f$ feature maps with $\alpha_a>0$, denoted $\tilde f^{(p)}$, and restricts all subsequent selection decisions to this positively contributing subset.

Redundancy between two filtered maps $p,q$ is measured by the absolute cosine similarity

$$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$0

A larger $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$1 indicates greater overlap in the information carried by the two maps. The feature-selection objective is then to choose a binary vector $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$2 such that the selected set maximizes overall importance $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$3 while penalizing pairwise redundancy $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$4, where

$$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$5

normalizes individual importances into $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$6. The term “importance” in the name of the method therefore refers specifically to class-conditioned feature-map importance derived from output gradients, rather than to importance weights in Monte Carlo estimation (Venturelli et al., 28 Apr 2026).

2. QUBO encoding and quantum-annealing dynamics

Each binary decision variable $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$7 is encoded as the projector

$$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$8

where $$f^{(a)}\in\mathbb{R}^{H_f\times W_f},\qquad a=1,\dots,N_f,$$9 is the Pauli-$z=\mathrm{CNN}(x)$0 operator on qubit $z=\mathrm{CNN}(x)$1. This yields the QUBO Hamiltonian

$z=\mathrm{CNN}(x)$2

The hyperparameter $z=\mathrm{CNN}(x)$3 controls the trade-off between favoring importance and penalizing redundancy. If $z=\mathrm{CNN}(x)$4, the model tries to select all positively contributing feature maps; if $z=\mathrm{CNN}(x)$5, only redundancy matters and the lowest-energy solution is the empty set. In the reported experiments, $z=\mathrm{CNN}(x)$6 yielded the best empirical disentanglement. The formulation contains no explicit cardinality constraint.

Quantum annealing is implemented by interpolating between a transverse-field driver Hamiltonian

$z=\mathrm{CNN}(x)$7

and the problem Hamiltonian $z=\mathrm{CNN}(x)$8. With the linear schedule

$z=\mathrm{CNN}(x)$9

the time-dependent Hamiltonian is

$a$0

At $a$1, the system starts in the uniform superposition $a$2, the ground state of $a$3. As $a$4 increases to $a$5, the dynamics are intended to track the ground state of $a$6 until the problem Hamiltonian encodes the selected feature subset.

For the small-scale simulations, the protocol used annealing times $a$7, a time step $a$8 for Schrödinger-equation integration, and $a$9 measurement shots per instance. At the end of the anneal, measurement in the computational basis yields a $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$0-bit string whose 1-entries indicate included feature maps. Repetition over many shots permits estimation of the ground-state success probability $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$1. The analysis also tracks the minimum spectral gap

$$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$2

because the adiabatic theorem implies that high-probability ground-state preparation requires $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$3 (Venturelli et al., 28 Apr 2026).

3. Empirical behavior on CNN interpretability tasks

The reported experiments used CIFAR-10-like data, specifically STL-10 with 10 classes, together with a ResNet-18 fine-tuned so that its final custom block had $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$4 feature maps. For each of $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$5 test images, the method produced a binary bit string from quantum-annealing-based feature selection. Aggregation over images of the same class yielded class-activity histograms, and pairwise Bhattacharyya coefficients between these discrete histograms produced a $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$6 class-class correlation map. A perfectly diagonal map corresponds to distinct feature-map subsets for distinct classes.

In the $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$7 setting, quantum-annealing feature selection produced only 15 non-zero overlaps out of the 45 possible off-diagonals, and all were below $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$8, indicating strong disentanglement. For the full ResNet-18 with $$\alpha_a = \frac{1}{H_f W_f}\sum_{i,j}\frac{\partial z}{\partial f_{ij}^{(a)}}.$$9, simulated annealing produced similarly diagonal maps, although full quantum-annealing-scale simulation was infeasible.

The method was also compared with GradCAM and GradCAM++ by translating the selected feature maps into coarse saliency masks through upsampling and measuring the “Average Drop %,” defined as the relative drop in class confidence when only the saliency-highlighted pixels are retained. Lower values are better.

Method	Setting	Average Drop %
GradCAM	$\alpha_a$0	$\alpha_a$1
GradCAM++	$\alpha_a$2	$\alpha_a$3
QA-fGradCAM ($\alpha_a$4)	$\alpha_a$5	$\alpha_a$6

These results place Importance Annealing below both classical baselines on this metric. In the full-block $\alpha_a$7 setting with simulated annealing, the Average Drop rose to approximately $\alpha_a$8, which the study attributes to the need to retune $\alpha_a$9 or introduce a cardinality penalty in very large-feature-map regimes (Venturelli et al., 28 Apr 2026).

4. Adiabaticity, fidelity, and scaling analysis

The reported theoretical and numerical analysis characterizes success in terms of the fidelity

$d\le N_f$0

between the final annealed state and the exact ground state. For $d\le N_f$1, the evolution is largely diabatic, with $d\le N_f$2, and the behavior scales roughly as $d\le N_f$3, consistent with the Landau-Zener form

$d\le N_f$4

For $d\le N_f$5, the process becomes effectively adiabatic and achieves $d\le N_f$6 for most images even up to $d\le N_f$7.

The minimum-gap distribution over all instances was reported as peaked near $d\le N_f$8 in dimensionless energy units, with almost zero probability below $d\le N_f$9. This was taken to imply that moderate annealing times suffice to avoid non-adiabatic transitions in the tested regime. A supplementary analysis further reported

$\alpha_a>0$0

and hence, through the adiabatic bound, $\alpha_a>0$1 suffices for high-fidelity ground-state preparation. The paper contrasts this polynomial scaling with the $\alpha_a>0$2 classical search cost and states that this illustrates a regime where quantum annealing offers an exponential-space speedup.

The coupling structure is also analyzed statistically. The $\alpha_a>0$3 values across random image instances follow an approximately Gaussian distribution, apart from a $\alpha_a>0$4-peak at zero for $\alpha_a>0$5, placing the problem in the “Sherrington-Kirkpatrick” universality class of mean-field spin glasses. At the same time, the study emphasizes that CNN-derived couplings are neither adversarially structured nor fully random. A plausible implication is that this intermediate structure contributes to the observed practicality of moderate-$\alpha_a>0$6 annealing in the tested instances (Venturelli et al., 28 Apr 2026).

5. Interpretability implications, outputs, and practical constraints

Importance Annealing produces what the study describes as bit-level explanations. Instead of aggregating all feature maps into a single coarse visualization, it identifies exactly which last-layer feature maps are included in the explanation for a particular prediction. These selected maps are then upsampled and summed as a saliency map for visualization. Aggregating the resulting bit strings across many examples yields class-level activity patterns and class-class disentanglement maps.

This output format supports several distinct interpretability claims made in the study. First, it isolates which learned kernels are active for a prediction rather than merely highlighting a spatial region. Second, it can expose shared or spurious features across classes by examining overlaps in selected subsets. Third, it motivates the possibility of CNN architectures whose interpretability is enforced through quantum-inspired regularization. These latter points are interpretive consequences drawn directly in the paper.

The practical pipeline is correspondingly structured. For each test image, one performs a forward pass to obtain the logit, a backward pass to compute $\alpha_a>0$7, filters the positive $\alpha_a>0$8, constructs normalized importance weights $\alpha_a>0$9 and redundancy couplings $\tilde f^{(p)}$0, chooses $\tilde f^{(p)}$1 and $\tilde f^{(p)}$2, anneals from $\tilde f^{(p)}$3 under $\tilde f^{(p)}$4, measures bit strings over $\tilde f^{(p)}$5 shots, and uses the selected subset to form the explanation map.

Several constraints follow from the reported evidence. There is no explicit cardinality constraint in the base formulation. Full quantum-annealing simulation was infeasible for $\tilde f^{(p)}$6. Performance in the large-feature-map setting degraded relative to the $\tilde f^{(p)}$7 custom block. These observations do not invalidate the method, but they delimit the regime in which the empirical evidence is strongest and suggest that hyperparameter retuning or modified objectives become important as $\tilde f^{(p)}$8 grows (Venturelli et al., 28 Apr 2026).

6. Relation to annealed importance sampling and other annealing frameworks

The term “Importance Annealing” should be distinguished from several established uses of annealing in statistics, Monte Carlo, and variational inference. In Annealed Importance Sampling, one anneals through a family of intermediate distributions

$\tilde f^{(p)}$9

and the central design issue is the variance of importance weights or log-weights in partition-function estimation (Kiwaki, 2015). In “Optimal schedules for annealing algorithms,” schedule design is formulated through an action functional involving a friction tensor $p,q$0, with optimal paths interpreted as geodesics in control-parameter space (Barzegar et al., 2024). In “Provable benefits of annealing for estimating normalizing constants,” the comparison is between annealed importance sampling, annealed NCE, and different interpolation paths such as geometric and arithmetic paths, with efficiency characterized by asymptotic mean-squared error (Chehab et al., 2023). In annealed variational inference, annealing instead introduces a temperature or inverse-temperature $p,q$1 into the objective $p,q$2 to mitigate mode collapse (Fogliani et al., 13 Feb 2026).

Importance Annealing differs from all of these uses in both its object of optimization and its meaning of “importance.” Here, “importance” refers to Grad-CAM-derived feature-map scores $p,q$3, not to importance weights, importance sampling, or normalizing-constant estimation. The annealing variable is likewise not a temperature over a statistical path, but the interpolation between a driver Hamiltonian and a QUBO Hamiltonian for feature selection. This terminological distinction is essential for situating the method within the broader annealing literature. At the same time, a plausible conceptual connection is that all of these frameworks use annealing to manage a trade-off between tractability at the beginning of the procedure and task fidelity at the end.