---
title: Attribution Gradients in ML Models
url: https://www.emergentmind.com/topics/attribution-gradients
type: topic
---

# Attribution Gradients in ML Models

Searching arXiv for the cited paper and related "attribution gradients" works to ground the article in current arXiv literature.
Attempting to query the arXiv search tool for the focal paper and nearby attribution-gradient literature.
Attribution gradients are gradient-based attribution methods that explain a scalar model output by differentiating it with respect to input features, embeddings, hidden states, or other intermediate representations, and then converting those derivatives into contribution scores. In the dominant machine-learning usage, the central object is a path integral of gradients between a baseline and an input, most notably in Integrated Gradients (IG); later work extends the same logic to dataset-wise summaries, transformer-layer decompositions, manifold- and distribution-aware paths, and covariance-filtered local explanations [1703.01365], [2404.13910], [2606.29693].

## 1. Foundational formalism

For a differentiable model \(F:\mathbb{R}^n \to \mathbb{R}\), attribution methods assign a relevance value to each input coordinate relative to a baseline \(x'\). Integrated Gradients uses the straight-line path \(\gamma(\alpha)=x'+\alpha(x-x')\) and defines
\[
\mathrm{IG}_i(x)=(x_i-x_i')\int_0^1 \frac{\partial F(x'+\alpha(x-x'))}{\partial x_i}\,d\alpha.
\]
Its practical importance comes from three properties emphasized in the literature: **Sensitivity**, **Implementation Invariance**, and **Completeness**, with
\[
\sum_i \mathrm{IG}_i(x)=F(x)-F(x').
\]
More generally, path methods replace the straight line by any smooth path \(\gamma\) from baseline to input and compute
\[
\mathcal{A}_i(\gamma)=\int_0^1 \frac{\partial F(\gamma(\alpha))}{\partial \gamma_i(\alpha)} \frac{\partial \gamma_i(\alpha)}{\partial \alpha}\,d\alpha.
\]
Within this class, the straight-line IG path is singled out as the unique path method that is symmetry-preserving [1703.01365].

A second foundational line unified several earlier attribution rules as modified gradients. In that framework, Gradient \(\times\) Input, IG, \(\epsilon\)-LRP, and DeepLIFT can all be written as feature-wise products of an input or input difference with either the true gradient or a modified gradient obtained by replacing activation derivatives with average slopes. This makes “attribution gradients” a family rather than a single formula: local gradients, path-integrated gradients, and modified backpropagated gradients differ mainly in where the derivative is evaluated and how it is averaged [1711.06104].

## 2. Path dependence, saturation, and weighting

A recurring theme is that attribution quality is path-dependent. In rectified deep networks, the gradient field is piecewise linear and highly discontinuous at region boundaries, so integrating along different paths can produce materially different explanations. One important critique goes further: for softmax classifiers, logit-gradients can be arbitrarily manipulated through shift-invariance of the logits without changing the discriminative function. This leads to the reinterpretation
\[
\nabla_x f_i(x)=\nabla_x \log p_\theta(x\mid y=i),
\]
so raw logit-gradients are better viewed as scores of an implicit class-conditional density model than as uniquely determined explanations of \(p_\theta(y\mid x)\). This suggests that the explanatory quality of gradients depends on how well that implicit density aligns with the data distribution, not only on discriminative accuracy [2006.09128].

A second major issue is **saturation** along the baseline-to-input path. Integrated Decision Gradients (IDG) addresses this by weighting each gradient by the derivative of the output logit with respect to the path parameter,
\[
IF(\alpha)=\frac{\partial F(x'+\alpha(x-x'))}{\partial \alpha},
\]
so attribution mass concentrates where the model actually “makes its decision.” The method introduces a path-specific Sensitivity axiom and uses adaptive sampling to allocate more evaluations to regions where the logit changes rapidly [2305.20052]. SmoothTaylor reaches a related destination from Taylor’s theorem: it averages first-order Taylor contributions from noisy roots near the input and can be viewed as a SmoothGrad-style special case where the smoothed quantity is \((x-z)\odot \nabla f(z)\), not merely \(\nabla f(x)\) [2004.10484].

Recent work has also attacked the baseline-weighting problem directly. Path-Sampled Integrated Gradients (PS-IG) defines attribution as the expected IG over baselines sampled along the linear interpolation path and proves equivalence to path-weighted IG when the weighting function equals the cumulative distribution function of the sampling density. In its deterministic form, this changes convergence from Monte Carlo \(O(m^{-1/2})\) to deterministic \(O(m^{-1})\), and under uniform sampling it strictly lowers attribution variance by a factor of \(1/3\) [2604.14338]. Weighted Integrated Gradients (WG) instead scores candidate baselines by a perturbation-based fitness criterion \(D_\alpha\), assigns non-uniform weights, and reports 10–35\% improvement over Expected Gradients on main metrics while also enabling baseline filtering to reduce computation [2505.03201].

## 3. Geometry-aware and modality-aware path design

A large branch of the literature keeps the IG axioms but replaces the straight-line path with one that is better matched to the modality or to model geometry. For perception tasks, Blur Integrated Gradients replaces intensity scaling by a Gaussian scale-space path \(L(x,y,\sigma)=G_\sigma * z\) and integrates
\[
\int_{\sigma=\infty}^{0}\frac{\partial F(L)}{\partial L}\frac{\partial L}{\partial \sigma}\,d\sigma.
\]
Because Gaussian blur satisfies scale-space causality and semigroup properties, this removes the need for a hand-chosen baseline in vision and audio settings and yields attributions in both space and scale/frequency [2004.03383].

For discrete language inputs, Uniform Discretized Integrated Gradients (UDIG) argues that straight-line interpolation in embedding space passes through points far from actual words. It therefore constructs a nonlinear path whose intermediate points stay close to real token embeddings and evaluates the resulting attributions on sentiment classification and question answering using Log odds, Comprehensiveness, and Sufficiency, outperforming existing methods on almost all metrics [2412.03886].

Other proposals impose geometric structure directly on the path. Spectral Integrated Gradients (SIG) decomposes the baseline-to-input difference by singular value decomposition and activates singular components from largest to smallest, producing a coarse-to-fine path that preserves completeness while reducing high-frequency attribution noise [2605.19607]. Manifold-Aligned Guided Integrated Gradients (MA-GIG) moves the path into the latent space of a pre-trained variational autoencoder so that decoded intermediate points are biased toward the learned generative manifold, which reduces off-manifold noise relative to input-space guidance [2605.02167]. FRInGe takes an even stronger geometric turn: it defines the reference in predictive distribution space as the maximum-entropy uniform distribution, follows a Fisher–Rao geodesic on the simplex, realizes the trajectory in input space via the pullback Fisher metric, and integrates gradients along that path; across six ImageNet architectures it most clearly improves calibration-oriented attribution metrics, especially MAS scores [2605.06404].

Diffusion Integrated Gradients (DiffIG) treats path generation itself as a conditional generative modeling problem. It trains a diffusion model on paths drawn from a Stick-Breaking Process, then uses guidance networks for faithfulness and complexity to steer sampling at inference time. The attribution rule remains the standard path integral, but the path is no longer fixed or hand-crafted; it is sampled from a controllable path distribution that can be optimized for perceptual alignment and perturbation faithfulness [2606.22314].

## 4. Layer-wise and probabilistic attribution in transformers

The most explicit layer-wise use of attribution gradients appears in IG-Lens, which asks: between which two layers is the probability of a predicted token actually produced? In decoder-only transformers with residual-stream states \(\mathbf{h}_0,\dots,\mathbf{h}_n\) and readout
\[
f(\mathbf{h})=\mathrm{softmax}(\mathrm{head}(\mathrm{norm}(\mathbf{h})))_{y_t},
\]
IG-Lens applies Integrated Gradients segment by segment between consecutive hidden states. For a chosen layer sequence \(L_0<\cdots<L_k\), the segment attribution is
\[
IG_{L_j}=\int_0^1 \nabla f\!\big(\mathbf{h}_{L_{j-1}}+a(\mathbf{h}_{L_j}-\mathbf{h}_{L_{j-1}})\big)\cdot(\mathbf{h}_{L_j}-\mathbf{h}_{L_{j-1}})\,da.
\]
By the gradient theorem,
\[
IG_{L_j}=f(\mathbf{h}_{L_j})-f(\mathbf{h}_{L_{j-1}}),
\]
and therefore
\[
\sum_{j=1}^{k} IG_{L_j}=f(\mathbf{h}_{\text{final}})-f(\mathbf{h}_{\text{base}}).
\]
This is telescoping completeness in **probability space**, with LayerNorm and softmax inside the path rather than linearized away [2606.29693].

This construction is motivated by the limits of earlier transformer readout tools. Logit lens and Tuned Lens provide per-layer probability levels but no additive decomposition; Direct Logit Attribution is exact in logit space but not in probability space because softmax is nonlinear; Layer Conductance computes integrated gradients per layer but changes the baseline across layers, so the layer scores do not telescope to the total probability change. IG-Lens resolves this by using a single path through intermediate hidden states and a prediction-aware estimator inspired by IDGI, under which each segment collapses to a telescoping sum of endpoint probabilities and completeness holds to floating point for any number of steps [2606.29693].

A common misconception is that exact additivity in logits can be read as exact additivity in probabilities. The literature is explicit that this fails once softmax is applied. IG-Lens therefore occupies a distinct niche: it attributes contributions directly to the scalar probability of a target token rather than to residual-stream logits alone [2606.29693].

## 5. Dataset-wise and tabular generalizations

Most attribution-gradient methods are instance-wise, but Integrated Gradient Correlation (IGC) extends the idea to datasets. Starting from per-sample IG values \(g_{i,j}(f,x_i,\bar{X})\), IGC defines a dataset-wise attribution for feature \(j\) by correlating those attributions with the centered target:
\[
h_j=\frac{1}{\sigma_{f(X)}\sigma_Y}\,\mathbb{E}\big[g_{i,j}(f,x_i,\bar{X})(y_i-\mu_Y)\big].
\]
The sum over features recovers the global Pearson correlation \(\rho_{f(X),Y}\), and additivity over regions of interest is immediate by summation. This makes IGC a dataset-level attribution-gradient method: it decomposes a global prediction score into per-feature contributions rather than explaining only one prediction at a time [2404.13910].

For tabular classifiers, AGOP-IxG modifies the local gradient itself. It computes the Average Gradient Outer Product
\[
M=\frac{1}{n}\sum_{i=1}^n g_i g_i^\top
\]
from training-sample gradients \(g_i\), truncates it to the top-\(K\) eigenspace \(M_K\), and then attributes using
\[
e_j(x)=(x_j-x'_j)\big(g(x)M_K\big)_j.
\]
This is exactly InputXGradient applied to a gradient pre-multiplied by a low-rank gradient-covariance filter. On synthetic linear, sparse nonlinear, and interaction-based tabular benchmarks with ground-truth per-sample attribution, AGOP-IxG leads on Spearman rank correlation and noise feature mass on all three synthetic datasets, and on top-\(k\) precision on the interaction dataset, while remaining approximately \(350\times\) to \(1{,}650\times\) faster than SHAP [2605.15700].

These dataset-wise and tabular variants underscore a broader point: attribution gradients are no longer confined to raw input saliency. They can summarize global strategies, enforce ROI additivity, or incorporate population-level gradient covariance before any per-feature score is formed. This suggests a shift from purely local sensitivity analysis toward structured attribution operators defined over examples, layers, or distributions.

## 6. Evaluation, limitations, and terminological ambiguity

Evaluation in this area is fragmented because different methods target different notions of fidelity. The literature uses **Sensitivity-n** to compare summed attributions against perturbation effects on subsets of features, **insertion/deletion**, **RISE**, **AIC**, **SIC**, **DiffID**, **MAS**, **infidelity**, **ROAR**, **IoU**, **Pointing Game**, **noise feature mass**, and simple completeness residuals [1711.06104], [2305.20052]. This metric diversity reflects a real methodological split: some methods optimize local faithfulness, some optimize calibration of attribution magnitude, and some optimize global feature ranking under retraining.

Several limitations recur across otherwise different proposals. Baseline dependence remains central for IG-style methods, whether the baseline is a zero vector, a blurred image, a mean hidden state, or a background sample [1703.01365]. Straight-line interpolation can traverse off-manifold or high-curvature regions, which motivates manifold-, scale-, spectral-, or distribution-aware paths [2004.03383]. Even exact completeness may not mean causal total effect: IG-Lens itself emphasizes that its head-once construction measures what norm+head can read out of \(\mathbf{h}_L\) beyond the previous chosen layer and therefore gives a conditional marginal readout rather than the total causal effect routed through higher blocks [2606.29693]. Dataset-wise aggregation also has caveats: IGC is most meaningful when the location of informative features is stable across the dataset, and negative IGC values do not mean irrelevance but systematic contribution opposite to target variation [2404.13910].

A deeper controversy concerns what gradients explain at all. The softmax shift-invariance result implies that raw input-gradients of logits are not uniquely tied to the discriminative function, so visually plausible saliency maps can arise from properties of an implicit generative model rather than from a canonical decomposition of \(p_\theta(y\mid x)\). This critique does not invalidate path-based attribution, but it does narrow what can be claimed from raw gradient structure alone [2006.09128].

Finally, the phrase *attribution gradients* has acquired a second meaning outside model attribution. In the HCI paper “Attribution Gradients: Incrementally Unfolding Citations for Critical Examination of Attributed AI Answers,” the term denotes an interface that incrementally expands an attributed answer into claims, supporting and contradictory excerpts, and nested citations. That usage is conceptually unrelated to gradient-based feature attribution, but it is now part of the arXiv record and makes the term formally ambiguous [2510.00361]. In current machine-learning practice, however, the dominant meaning remains the one developed from Integrated Gradients: gradients or integrated gradients of a scalar readout, accumulated along a carefully chosen path to yield an attribution of model behavior.

Source: https://www.emergentmind.com/topics/attribution-gradients