Delta-Attribution: Explaining Model Updates

• Delta-Attribution is a model-agnostic framework that quantifies changes in feature reliance by differencing local attributions between model versions.
• It computes attribution deltas using occlusion in a standardized feature space, isolating the effects of hyperparameter and data updates.
• The approach leverages metrics like BAC, DCE, and JSD to distinguish benign tweaks from significant shifts, aiding in model update auditing.

Delta-Attribution denotes a model-agnostic framework for explaining what changed between model versions $A$ and $B$ by differencing per-feature local attributions, $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$. Introduced in “Delta-Audit: Explaining What Changes When Models Change,” it is aimed at update auditing rather than single-model explanation: the objective is to expose redistribution of feature reliance under changes in hyperparameters, kernels, tree depth, solvers, search procedures, or training data, and to relate those reliance shifts to observed behavioural change (Hemmat et al., 27 Aug 2025). In broader attribution theory, completeness is often expressed as a decomposition of a prediction difference $f(x)-f(x')$, which the literature already treats as a baseline-delta object; Delta-Attribution extends that difference-based perspective from input baselines to model-version comparisons (Taimeskhanov et al., 30 May 2025).

1. Definition and conceptual scope

The central definition is

$\Delta\phi(x)=\phi_B(x)-\phi_A(x),$

where $A$ and $B$ are two versions of the same predictive pipeline, $\phi_A(x)\in\mathbb{R}^d$ and $\phi_B(x)\in\mathbb{R}^d$ are per-feature attribution vectors returned by some explainer $E$, and $B$0 is interpreted as the redistribution of reliance over features induced by the update from $B$1 to $B$2 (Hemmat et al., 27 Aug 2025).

This definition fixes the topic’s main distinction from ordinary feature attribution. A single-model explanation describes what one model currently uses. Delta-Attribution asks what changed between versions and whether that change aligns with the observed output difference. The paper’s formulation is explicitly operational: models are frequently updated by changing hyperparameters, kernels, tree depth, solvers, search procedures, or data, and accuracy alone does not reveal why behaviour changed, whether the change is benign, or whether reliance has shifted toward or away from task-relevant signals (Hemmat et al., 27 Aug 2025).

A common misconception is that Delta-Attribution is a new standalone explainer. The framework is instead described as a way to “turn any local explainer into an update explainer” by differencing explanations across versions. Another misconception is that it replaces performance evaluation. The intended role is complementary: it supplements accuracy, calibration, or error analyses by exposing version-to-version changes in feature reliance (Hemmat et al., 27 Aug 2025).

2. Computational formulation

In the paper’s instantiation, Delta-Attribution is implemented with fast occlusion/clamping in standardized feature space. Two classifiers $B$3 and $B$4 are compared over inputs $B$5. For each input, the reference class $B$6 is fixed as the class predicted by the updated model $B$7. The class score is then defined as a class-anchored margin or log-odds: $B$8 Anchoring to $B$9, the class predicted by $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$0, is reported to stabilize the behaviour-linked metrics, particularly BAC and DCE (Hemmat et al., 27 Aug 2025).

Per-feature attribution is computed by clamping one standardized feature at a time to a shared baseline vector $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$1: $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$2 The same baseline $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$3 and the same reference class $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$4 are used for both $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$5 and $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$6, so that

$\Delta\phi(x)=\phi_B(x)-\phi_A(x)$7

isolates model-version differences rather than explanation-setup differences. The baseline is derived from the training set, mean by default and optionally averaged with the median to reduce baseline artifacts. Because all features are standardized first via a StandardScaler, clamping is performed in standardized space (Hemmat et al., 27 Aug 2025).

The behavioural quantity paired with $\Delta\phi(x)=\phi_B(x)-\phi_A(x)$8 is

$\Delta\phi(x)=\phi_B(x)-\phi_A(x)$9

the change in class-anchored score between versions. The underlying intuition is direct: if the update changes model behaviour strongly on a sample, a useful update explainer should reveal a correspondingly large or otherwise meaningful redistribution of feature reliance on that sample (Hemmat et al., 27 Aug 2025).

3. The Delta-Attribution Quality Suite

The paper evaluates $f(x)-f(x')$0 with a “Delta-Attribution Quality Suite” organized into four categories: magnitude/sparsity, agreement/shift, behavioural alignment, and robustness. Let

$f(x)-f(x')$1

skipping samples with $f(x)-f(x')$2. Averages are taken over the test set or a stratified subset of up to 256 samples (Hemmat et al., 27 Aug 2025).

Magnitude is measured by

$f(x)-f(x')$3

reported as $f(x)-f(x')$4Mag$f(x)-f(x')$5 or Mag$f(x)-f(x')$6. It quantifies the overall size of the reliance change. Concentration is measured with Top-$f(x)-f(x')$7 mass and entropy. Top-$f(x)-f(x')$8 concentration is

$f(x)-f(x')$9

and entropy is

$\Delta\phi(x)=\phi_B(x)-\phi_A(x),$0

High Top-$\Delta\phi(x)=\phi_B(x)-\phi_A(x),$1 mass and low entropy indicate that the update reallocated reliance across a small number of features (Hemmat et al., 27 Aug 2025).

Agreement and redistribution between $\Delta\phi(x)=\phi_B(x)-\phi_A(x),$2 and $\Delta\phi(x)=\phi_B(x)-\phi_A(x),$3 are measured with RankOverlap@10 and Jensen–Shannon divergence. RankOverlap@10 is the Jaccard overlap of $\Delta\phi(x)=\phi_B(x)-\phi_A(x),$4 and $\Delta\phi(x)=\phi_B(x)-\phi_A(x),$5. JSD is computed after normalizing absolute attributions: $\Delta\phi(x)=\phi_B(x)-\phi_A(x),$6 and reporting

$\Delta\phi(x)=\phi_B(x)-\phi_A(x),$7

Low JSD indicates similar distribution of attribution mass; high JSD indicates genuine redistribution of reliance (Hemmat et al., 27 Aug 2025).

Behavioural alignment is captured by Delta Conservation Error and Behaviour–Attribution Coupling: $\Delta\phi(x)=\phi_B(x)-\phi_A(x),$8

$\Delta\phi(x)=\phi_B(x)-\phi_A(x),$9

DCE is an additive-consistency diagnostic, with smaller values better. BAC is the Pearson correlation across samples between attribution-delta magnitude and absolute behaviour change. High BAC means samples with larger output changes also exhibit larger reliance shifts (Hemmat et al., 27 Aug 2025).

The suite also includes CO$A$0F, class-outcome focus, described as the fraction of $A$1-mass on the top-$A$2 globally relevant features for $A$3 from permutation importance, with $A$4, reported separately for fixes and regressions. Robustness includes Gaussian-noise sensitivity,

$A$5

baseline sensitivity, and grouped occlusion. For grouped occlusion, the paper jointly clamps the top-$A$6 features by $A$7, recomputes $A$8, and reports

$A$9

using $B$0 in the experiments (Hemmat et al., 27 Aug 2025).

4. Experimental program

The empirical study covers 45 audited settings: five classical model families, three datasets, and three A/B pairs per family (Hemmat et al., 27 Aug 2025). The datasets are Breast Cancer ($B$1, $B$2, binary classification), Wine ($B$3, $B$4, 3-class), and Digits ($B$5, flattened $B$6 images, $B$7, 10-class), each with a stratified 80/20 train-test split and shared preprocessing through StandardScaler and a common Pipeline (Hemmat et al., 27 Aug 2025).

The audited learner families are Logistic Regression, SVC, Random Forests, Gradient Boosting, and $B$8NN. The A/B settings were chosen to span both substantive inductive-bias changes and small implementation tweaks. For example, Logistic Regression compares changes in $B$9, L1 versus L2 regularization, and solver changes; SVC compares linear, poly, and rbf kernels as well as $\phi_A(x)\in\mathbb{R}^d$0 versus $\phi_A(x)\in\mathbb{R}^d$1; Random Forests vary estimator count, depth, and $\phi_A(x)\in\mathbb{R}^d$2; Gradient Boosting varies learning rate, estimator count, and depth; $\phi_A(x)\in\mathbb{R}^d$3NN varies $\phi_A(x)\in\mathbb{R}^d$4, weighting, and search algorithm (Hemmat et al., 27 Aug 2025).

This design is significant because Delta-Attribution is meant to detect when an update is merely cosmetic and when it changes model behaviour through a genuine redistribution of reliance. The A/B grid therefore functions as an update taxonomy: some pairs are expected to change inductive bias, while others are expected to leave the effective attribution structure almost unchanged (Hemmat et al., 27 Aug 2025).

5. Empirical patterns

The paper’s central empirical claim is that inductive-bias changes yield large, behaviour-aligned deltas, while cosmetic tweaks yield near-zero or rank-preserving deltas. The abstract’s representative examples are support vector machines on Breast Cancer and random forests on Digits. For SVC poly$\phi_A(x)\in\mathbb{R}^d$5rbf on Breast Cancer, the reported values are $\phi_A(x)\in\mathbb{R}^d$6 and $\phi_A(x)\in\mathbb{R}^d$7. For a Random Forest feature-rule swap on Digits, the reported values are $\phi_A(x)\in\mathbb{R}^d$8 and $\phi_A(x)\in\mathbb{R}^d$9. These are presented as cases of large and behaviour-aligned reliance reallocation (Hemmat et al., 27 Aug 2025).

By contrast, “cosmetic” tweaks are characterized by perfect or near-perfect agreement metrics. The abstract highlights SVC $\phi_B(x)\in\mathbb{R}^d$0 versus $\phi_B(x)\in\mathbb{R}^d$1 and $\phi_B(x)\in\mathbb{R}^d$2NN search changes as cases with RankOverlap@10 $\phi_B(x)\in\mathbb{R}^d$3 and $\phi_B(x)\in\mathbb{R}^d$4. The fuller results list five A/B pairs with $\phi_B(x)\in\mathbb{R}^d$5 and five with RankOverlap@10 $\phi_B(x)\in\mathbb{R}^d$6, including digits–knn–pair3, breast_cancer–svc–pair2, wine–knn–pair3, wine–rf–pair3, and breast_cancer–knn–pair3. These cases serve as sanity checks: the framework does not treat every parameter change as behaviourally meaningful (Hemmat et al., 27 Aug 2025).

The strongest redistribution in attribution mass is reported for deeper Gradient Boosting on Breast Cancer, with $\phi_B(x)\in\mathbb{R}^d$7. Other high-JSD cases include Breast Cancer Logistic Regression pair2 at approximately $\phi_B(x)\in\mathbb{R}^d$8, Wine $\phi_B(x)\in\mathbb{R}^d$9NN pair1 at approximately $E$0, Digits Random Forest pair2 at approximately $E$1, and Breast Cancer $E$2NN pair1 at approximately $E$3. This indicates that redistribution and behavioural coupling are distinct dimensions: an update can strongly redistribute reliance without being the strongest in BAC (Hemmat et al., 27 Aug 2025).

Algorithm-level summaries reinforce this distinction. Random Forests exhibit the largest average shifts with strong average coupling: $E$4 $E$5NN also shows substantial average shift,

$E$6

but with markedly weaker behavioural coupling. Logistic Regression is comparatively stable,

$E$7

SVC has small average deltas but moderate behaviour relevance, and Gradient Boosting falls between the linear and tree-based extremes (Hemmat et al., 27 Aug 2025).

These findings support a three-way interpretive scheme. Some updates are benign because effective reliance remains unchanged. Some are behaviour-aligned because changed outputs are matched by changed reliance. Others, especially several $E$8NN changes, show large but diffuse deltas with weaker BAC, indicating that a model can move substantial attribution mass without cleanly explaining its own behavioural shift (Hemmat et al., 27 Aug 2025).

6. Practical interpretation, limitations, and relation to broader attribution theory

The framework is presented as an audit tool for CI, regression testing, and model governance. The paper proposes practical heuristics for tabular settings with $E$9. A “benign update” is characterized by $B$00, $B$01 in the bottom quartile, RankOverlap@10 $B$02, and $B$03. A “behaviour-aligned shift” has $B$04 with medium or large $B$05. A “risky shift” has $B$06 with medium or large $B$07 or $B$08. The authors explicitly present these as heuristics rather than universal thresholds (Hemmat et al., 27 Aug 2025).

Several limitations are central to the concept. First, Delta-Attribution is explainer-dependent. In the paper’s implementation, the explainer is occlusion/clamping, which is not additive. For that reason, DCE is described as diagnostic rather than something expected to be zero. High DCE may indicate feature interactions or limitations of the explainer rather than a useless update explanation. The paper specifically notes that path methods such as Integrated Gradients do satisfy additivity and may be preferable when additive faithfulness is essential (Hemmat et al., 27 Aug 2025).

Second, baseline choice matters. The authors reduce baseline sensitivity by averaging mean and median training-set baselines when possible, but they note that extreme distributions may require domain-specific baselines. Third, correlated or interacting features remain a concern, which is why grouped occlusion is included as a robustness stress test. Fourth, the empirical scope is limited to classical tabular and low-dimensional image-like features; the study does not establish the same behaviour for very high-dimensional settings such as text, vision, or LLMs (Hemmat et al., 27 Aug 2025).

In broader attribution research, Delta-Attribution occupies a specific niche. Standard feature attribution explains a prediction within one model. Theoretical work such as “Feature Attribution from First Principles” emphasizes that many attribution frameworks already revolve around difference objects such as $B$09 or more general increment-based decompositions (Taimeskhanov et al., 30 May 2025). Delta-Attribution takes that logic from input-side differences to model-version differences: the explained object is not a single prediction, but the change in reliance profile induced by replacing model $B$10 with model $B$11.

The resulting concept is therefore neither a replacement for accuracy nor a replacement for attribution. It is an update-centric layer above both. Its value lies in distinguishing updates that are merely cosmetic from updates that meaningfully alter how a model uses features, and in doing so with a metric suite that jointly measures magnitude, redistribution, behavioural coupling, and robustness (Hemmat et al., 27 Aug 2025).