---
title: 'EAP-IG: Adaptive Integrated Gradients'
url: https://www.emergentmind.com/topics/integrated-gradients-eap-ig
type: topic
---

# EAP-IG: Adaptive Integrated Gradients

Integrated Gradients (EAP-IG) generalizes the classical integrated gradients (IG) attribution framework by replacing the uniform integration (Riemann) measure along the straight-line path with a non-uniform or adaptive weighting. This construct expands the expressive capacity of path-based attributions, allowing more flexible and potentially data- or geometry-adaptive integration schemes. EAP-IG (alternatively, "Expected Adaptive Path–IG," *Editor's term*) maintains the axiomatic core of IG under mild conditions, but diverges from standard uniqueness characterizations. Several recent directions—including its application to manifold alignment [2503.08240], faithfulness in circuit discovery [2403.17806, 2510.00845], and weighted Riemann Opt approaches [2410.04118, 2306.13753]—explore the theoretical and practical impact of EAP-IG.

## 1. Mathematical Definition of EAP-IG

Consider a differentiable function $f : \mathbb{R}^n \to \mathbb{R}$, a sample $x \in \mathbb{R}^n$ to be explained, and a baseline $x' \in \mathbb{R}^n$. Standard IG is defined by
\[
\mathrm{IG}_i(x; x') = (x_i - x'_i) \int_{0}^1 \frac{\partial f(x' + t(x-x'))}{\partial x_i} dt
\]
EAP-IG generalizes this by introducing a non-uniform weight function $w : [0,1] \to \mathbb{R}_+$ with $\int_0^1 w(t)\,dt=1$, yielding
\[
\mathrm{EAP}\_\mathrm{IG}_i(x; x') = (x_i - x'_i) \int_0^1 w(t) \frac{\partial f(x' + t(x-x'))}{\partial x_i} dt
\]
where $w$ encodes the preference for sampling at particular locations along the path.

## 2. Axiomatic Properties and Uniqueness

EAP-IG preserves the completeness, linearity, implemented invariance, non-decreasing positivity, symmetry, and affine scale invariance axioms inherited from classical IG, so long as $w(t)\geq 0$ and $\int_0^1 w(t)\,dt=1$ [2306.13753]. Explicitly:
- **Completeness**: $\sum_i \mathrm{EAP}\_\mathrm{IG}_i = f(x) - f(x')$, since
  \[
  \sum_i \int_0^1 w(t)\,\nabla f(\gamma(t))\cdot\dot\gamma(t)\,dt = \int_0^1 w(t)\frac{d}{dt} f(\gamma(t))dt = f(x) - f(x')
  \]
  for any straight-line path $\gamma$.
- **Positivity**: If $f$ is non-decreasing along the path, positivity holds coordinatewise.
- **Symmetry-Preserving**: If $f$ is invariant under swapping coordinates and $x,x'$ respect this, then $\mathrm{EAP}\_\mathrm{IG}_i=\mathrm{EAP}\_\mathrm{IG}_j$ for $i,j$.

However, uniqueness results that single out the uniform weight ($w\equiv 1$) depend on an additional reparameterization-invariance axiom. Any non-constant $w$ (yielding EAP-IG) always preserves the classical componentwise axioms, but violates reparameterization invariance [2306.13753].

## 3. Algorithmic Variants and Optimization of Integration Paths

The EAP-IG variant enables explicit optimization over the path integral weighting or support. This is operationalized via:
- **Weighted Riemann sampling**: One chooses $\{\alpha_j\}_{j=1}^m$ not uniformly, but to minimize integral discretization error via a data-driven criterion, as in RiemannOpt [2410.04118]. The optimal breakpoints can be found by minimizing
  \[
  \sum_{i=0}^{N-1} w_i (\alpha_{i+1} - \alpha_i)^2
  \]
  where $w_i$ estimates the absolute derivative of the integrand at $\alpha_i$.
- **Tangentially Aligned Integrated Gradients**: The baseline $x'$ can be optimized to maximize tangential alignment relative to the data manifold, yielding attributions lying in the manifold tangent space [2503.08240].
- **Adaptive path selection in circuit mechanisms**: In EAP-IG for circuit interpretability [2403.17806, 2510.00845], weights or sampling locations are tailored to reflect intervention relevance or causal saliency.

| EAP-IG instantiation         | Weighting scheme / path               | Primary function       |
|------------------------------|---------------------------------------|------------------------|
| Uniform (standard IG)        | $w(t)=1$                              | Canonical path        |
| Data-driven RiemannOpt       | $w$ via optimization of integrand var | Noise/error reduction |
| Tangential alignment         | $w$ induced by manifold geometry      | Human-aligned support |
| Mechanism/circuit focus      | $w$ reflects intervention range       | Causal faithfulness   |

## 4. Faithfulness, Variance, and Interpretability in Mechanistic Discovery

EAP-IG models are central to recent progress in finding faithful circuit representations in large neural transformers:
- Faithfulness is defined as the preservation of task-specific performance after ablating all edges outside the discovered subgraph [2403.17806]. EAP-IG yields circuits significantly more faithful (i.e., closer to clean model behavior) than vanilla EAP methods, especially at small circuit sizes. This is due to EAP-IG's avoidance of zero-gradient pathologies.
- Recent work has framed EAP-IG circuits as statistical estimators, assessing structural and performance variance under multiple perturbations [2510.00845]. High variance and hyperparameter sensitivity, such as to the number of interpolation steps or intervention schemes, have been empirically demonstrated, necessitating routine reporting of stability metrics.

Key stability metrics include:
- Circuit error (mean classification divergence)
- Jaccard index (structural edge overlap variance)
- Response under prompt paraphrasing, data resampling, and controlled random ablation

Best practices now recommend:
- Reporting mean/variance of faithfulness and structure under resampling
- Explicit justification and sensitivity sweeps of EAP-IG settings (aggregation, intervention choice)
- Noise injection stress-testing to reveal instability modes

## 5. EAP-IG in Manifold-Constrained and Tangent-Space-Optimized Attribution

EAP-IG supports geometric regularization of the baseline and the integration path:
- In Tangentially Aligned Integrated Gradients (TA-IG), the baseline $x'$ is selected so that the resultant attribution vector is maximally aligned to the tangent space $T_xM$ of an empirical data manifold $M\subset\mathbb{R}^d$ [2503.08240].
- The tangential-alignment score $\mu_x(v)=\| \pi_x v \|^2 / \| v \|^2$ formalizes this principle: the optimal baseline solver seeks $\alpha^*_x = \arg\min_{\alpha\in M,\,\alpha\neq x} \frac{1}{2}\|\pi^\perp_x \mathrm{IG}(x,\alpha)\|^2$.
- Empirically, TA-IG yields attributions much more concentrated in perceptually meaningful, manifold-supported directions than any standard baseline across several image datasets (e.g., $\mu>0.9$ vs. $\sim0.2$ for common baselines).

## 6. Limitations and Open Issues

EAP-IG introduces new classes of hyperparameters and potential sources of instability:
- The optimality of a weight $w$ is context- and task-dependent. Faithfulness or interpretability gains may be offset by sensitivity to the choice of $w$, the method for baseline selection, or the geometry of the manifold encoder.
- Convergence of implicit optimization (e.g., for tangent alignment) is not guaranteed in nonconvex regimes; local minima can yield only approximately tangential attributions.
- Computational cost increases linearly with the number of integration steps (for most implementations), and for manifold optimization, further overhead arises from tangent estimation.
- The explanatory utility of EAP-IG variants is bounded by the quality of the generative/discriminative manifold model and the faithfulness of surrogates in physical-design tasks.

Rigid axiomatic uniqueness is only preserved for uniform $w$; deviations require careful justification in each context.

## 7. Practical Guidelines

For effective use of EAP-IG:
- For circuit discovery in transformers, use $m=3\ldots5$ interpolation steps and greedily expand the subgraph until $1{-}2$\% edge coverage or the target normalized faithfulness is achieved [2403.17806].
- When optimizing Riemann weights for noise minimization, precompute breakpoints on a representative validation subset and reuse for bulk attribution [2410.04118].
- For tangentially aligned IG, set latent dimensionality of the autoencoder according to observed data manifold rank, and apply regular projection to keep optimized baselines on manifold [2503.08240].
- Always report circuit faithfulness, Jaccard overlap, and performance variance under data and hyperparameter perturbations, and perform robustness checks with noise injection [2510.00845].
- In image-based tasks, use high percentile clipping of saliency maps and threshold overlays to reveal semantically meaningful attributions; in manifold-constrained settings, validate alignment by measuring the tangentiality score $\mu_x(\mathrm{IG})$.

EAP-IG’s generalization capacity enables tailored attribution design—either for geometric priors, causal science, or improved noise and faithfulness—which can be further specialized through data- or task-adaptive weighting of the IG integral. This flexibility makes EAP-IG foundational to current and emerging explainability methodologies in high-dimensional, structured domains.

Source: https://www.emergentmind.com/topics/integrated-gradients-eap-ig