---
title: Integrated Gradients Overview
url: https://www.emergentmind.com/topics/integrated-gradients
type: topic
---

# Integrated Gradients Overview

Integrated Gradients (IG) is a path-based feature attribution method for neural networks and other differentiable models, introduced to rigorously decompose the output difference between an input and a user-defined "baseline" into per-feature contributions. IG has become a canonical technique in explainable artificial intelligence (XAI) due to its rigorous axiomatic foundations and practical flexibility. The method integrates gradients of the model output with respect to the input along a straight path from the baseline to the input, yielding attributions that satisfy completeness, sensitivity, and implementation invariance. IG has inspired a broad family of path-integral attribution techniques, several formal uniqueness results, and a range of variants adapted to different data modalities, model pathologies, and robustness requirements.

## 1. Theoretical Foundation and Core Axioms

Integrated Gradients is formulated for a differentiable function $F:\mathbb{R}^n \rightarrow \mathbb{R}$, a target input $x$, and a reference baseline $x'$. The canonical IG attribution for feature $i$ is
\[
\mathrm{IG}_i(x) = (x_i - x'_i)\int_{0}^{1} \frac{\partial F\big(x' + \alpha(x - x')\big)}{\partial x_i}d\alpha.
\]
Central to IG's design are several axioms, which it uniquely satisfies among a large class of attribution methods [1703.01365, 2306.13753]:
- **Completeness**: $\sum_i \mathrm{IG}_i(x) = F(x) - F(x')$.
- **Implementation Invariance**: If two models compute the same function, their IG attributions match.
- **Linearity**: For $aF + bG$, attributions are $a$\,$\mathrm{IG}[F]+\,$$b$\,$\mathrm{IG}[G]$.
- **Sensitivity/Dummy**: Features that do not affect the output across the path receive zero attribution.

For analytic or piecewise-analytic functions, Integrated Gradients is characterized as the unique single-path method that satisfies completeness, symmetry, linearity, affine scale invariance, and proportionality among monotone straight-line paths [2306.13753]. This formal grounding distinguishes IG from heuristic or partial approaches such as input$\times$gradient, DeepLIFT, or Layerwise Relevance Propagation, which may violate key axioms.

## 2. Methodology, Computation, and Implementation

In practice, $F$ is a deep network for which the path-integral above lacks a closed form. IG is computed by discretizing the interval $\alpha\in[0,1]$ (typically into $20$–$300$ steps), evaluating the gradient at each interpolation point, and averaging:
\[
\mathrm{IG}_i(x) \approx (x_i - x'_i)\cdot \frac{1}{m} \sum_{k=1}^{m} \frac{\partial F(x' + \frac{k}{m}(x - x'))}{\partial x_i}.
\]
Computational cost scales linearly with the number of steps, as each requires a forward and backward pass. The completeness property can be numerically checked to monitor integration and numerical error.

Empirical usage requires careful baseline selection, as $x'$ must correspond to genuine absence of signal (e.g., black image, zero embedding). Inappropriate baselines can introduce bias or spurious attributions, though the method retains formal guarantees relative to any fixed baseline [1703.01365].

## 3. Limitations and Pathologies: Saturation, Baseline Sensitivity, and Discrete Domains

Despite its axiomatic guarantees, IG faces practical challenges:
- **Saturation effect**: For many deep networks, outputs may saturate (i.e. plateau) along the input path before reaching the target; gradients in these regions are near zero, yet IG weights all path segments equally. This can lead to "noisy" or incomplete attributions dominated by uninformative regions [2010.12697, 2305.20052].
- **Baseline dependence**: The choice of $x'$ can drastically affect attributed features; standard choices (zero, mean, blurred) are not universally "neutral" [2310.04821, 2503.08240].
- **Discrete or off-manifold interpolation**: For data such as word embeddings or graphs, the straight path from $x'$ to $x$ may traverse non-data-manifold, out-of-distribution regions. Gradients computed there do not reflect semantically meaningful changes [2108.13654, 2412.03886, 2509.07648].

Several approaches address these issues, including alternative path constructions, multiple baselines, and discretization strategies. Notably, adaptive sampling and non-uniform Riemann sums can reduce integration error by concentrating steps in information-rich path segments [2410.04118, 2305.20052].

## 4. Variants and Methodological Extensions

A diverse array of IG extensions adapt the method to domain-specific requirements, address noise/saturation, or generalize the attribution framework:
- **Integrated Decision Gradients (IDG)**: Weights each pathwise gradient by the derivative of the output logit, focusing on decision regions and eliminating saturated-region contributions. IDG combines an importance factor $\frac{\partial F}{\partial \alpha}$ with adaptive sampling along the path, empirically yielding sharper, more informative attributions [2305.20052].
- **Path-Weighted IG (PWIG)**: Generalizes the IG integral by assigning a user-defined weight $w(\alpha)$ to emphasize or de-emphasize specific path regions. PWIG enables focus on early, late, or intermediate path segments, at the expense of completeness except when $w\equiv1$ [2509.17491].
- **Guided IG (GIG)**: Employs adaptive paths that avoid high-noise, off-object regions by greedily moving along low-sensitivity features and optionally anchoring steps to the straight-line path, significantly reducing noise artifacts in vision models [2106.09788].
- **Counterfactual and Shapley-inspired Baselines**: Multiple or data-driven baselines (e.g., Shapley IG) reduce baseline-dependence and align IG with Shapley value theory, achieving improved faithfulness to true feature contributions [2310.04821].
- **Manifold and Geodesic IG**: On Riemannian manifolds (images, embeddings), IG along geodesics (e.g. as in GIG [2502.12108] or Manifold IG [2405.09800]) produces attributions that align with intrinsic data geometry, reducing spurious attributions and increasing robustness to adversarial perturbations.
- **Graph and Discretized IG**: On non-Euclidean domains, IG is adapted to sum over meaningful discrete paths (e.g., all shortest paths in a graph [2509.07648]) or snap interpolants to actual vocabulary embeddings (e.g., DIG, UDIG [2108.13654, 2412.03886]) to ensure path points are data-valid.

A summary of selected variants and domain adaptations is given below:

| Variant        | Key mechanism                     | Application area   |
|----------------|----------------------------------|--------------------|
| IDG            | Importance factor, adaptive grid  | Vision             |
| GIG            | Data/model-guided path            | Vision             |
| PWIG           | Path-dependent weights $w(\alpha)$| Medical imaging    |
| DIG/UDIG       | Monotonic/nearest-word paths      | NLP embedding      |
| Manifold IG    | Geodesics in latent/data manifold | Vision, structured |
| Graph IG       | Discrete shortest path sums       | GNNs, graphs       |
| SIG            | Shapley-weighted baseline avg     | General            |

## 5. Benchmarking and Empirical Results

Standard IG and its extensions have been evaluated across vision, NLP, medical imaging, and GNNs, using datasets such as ImageNet, SST-2, OASIS-1, and ShapeGGen. Evaluation metrics include:
- **Insertion/Deletion score (RISE AUC)**: Measures how attributions correspond to true impact on model output.
- **Softmax and Accuracy Information Curve AUCs**: Evaluate attributions via partial image/text reveals.
- **Faithfulness (ABPC, LO, Comprehensiveness/Sufficiency)**: Quantifies the effect of ablating or keeping top-$k$ attributed features [2010.12697, 2509.17491, 2412.03886].
- **Sharpness, stability, human alignment**: Both qualitative and quantitative (variance under input noise, agreement with human annotators).

IDG demonstrated consistently higher RISE and SIC/AIC insertion AUCs, sharper heatmaps, and lower spurious activations compared to IG, Left-IG, GIG, and Adversarial GI, with up to $5$–$15\,\%$ improvement [2305.20052]. Variants such as GIG, DIG, and Manifold IG have outperformed vanilla IG by similar margins across relevant domain-specific metrics [2106.09788, 2405.09800, 2412.03886].

## 6. Interpretability, Robustness, and Limitations

Integrated Gradients and its variants provide interpretable decompositions rooted in clear methodology, but their faithfulness depends on path, baseline, and domain alignment. While the original IG method is vulnerable to saturation, path-length cancellation, baseline ambiguity, and non-manifold interpolation, state-of-the-art approaches (IDG, GIG, PWIG, Manifold IG) offer targeted mitigation strategies. However, these often increase computational cost or weaken completeness, and introduce hyperparameter or model structure dependencies. The approximation of the path integral via Riemann sums remains a source of bias unless sample placement is adaptively optimized [2410.04118].

A plausible implication is that "hybrid" schemes—combining model-based path adaptations, manifold-aware integration, and Shapley-style baseline ensembles—will be essential for trustworthy attributions in high-dimensional, multimodal, or safety-critical settings.

## 7. Future Directions

Active research targets several open directions:
- **Learning manifold-conforming or data-aware paths** via generative models, adaptive weighting, or variational optimization [2405.09800, 2502.12108].
- **Algorithmic complexity reduction** through optimized Riemann sum scheduling, batchwise integration, and instance-conditional path choice [2410.04118].
- **Extending completeness and axiomatic properties** to non-uniform and non-monotonic path methods.
- **Cross-domain and structured-data adaptations** for GNNs, time series, and multimodal data [2509.07648].
- **Benchmarking via causal and human-centered metrics**, to calibrate attribution faithfulness in the absence of ground truth.

Integrated Gradients remains the canonical reference point for axiomatic, path-based attributions in XAI, with its methodological descendants advancing both theoretical and practical rigor across application areas [1703.01365, 2306.13753, 2305.20052].

Source: https://www.emergentmind.com/topics/integrated-gradients