---
title: 'Wasserstein-RIF: Robust Data Attribution'
url: https://www.emergentmind.com/topics/wasserstein-robust-influence-functions-w-rif
type: topic
---

# Wasserstein-RIF: Robust Data Attribution

Wasserstein-Robust Influence Functions (W-RIF) generalize classical influence functions to provide certified robustness under distributional shifts, using optimal transport metrics. W-RIF enables the quantification of how training examples influence model predictions while accounting for worst-case perturbations measured in Wasserstein distance. This framework yields formal coverage guarantees in convex models and provides new geometric tools for certified data attribution in deep neural networks, overcoming severe limitations of Euclidean-based certification by introducing a Natural Wasserstein metric derived from feature covariance geometry [2512.09103].

## 1. Classical Influence Functions and Their Limitations

Given a data-generating distribution $P$ over $\mathcal{Z}$ and a twice-differentiable loss $L(\theta;z)$, the empirical risk minimizer is
\[
\hat{\theta} = \arg\min_{\theta\in\mathbb{R}^p} \;\mathbb{E}_{z\sim P_n}[L(\theta;z)]
\]
for empirical $P_n = \frac{1}{n}\sum_{i=1}^n \delta_{z_i}$. The classical influence of training point $z_i$ on test loss at $z_{\text{test}}$ is
\[
\mathcal{I}(z_i, z_{\text{test}}) = -\nabla_\theta L(\hat{\theta}; z_{\text{test}})^\top H^{-1} \nabla_\theta L(\hat{\theta}; z_i)
\]
where $H = \mathbb{E}_{P_n}[\nabla^2_\theta L(\hat{\theta};z)]$ is assumed positive definite. This formula quantifies the first-order impact of up-weighting or removing $z_i$ but lacks robustness to distributional perturbations and fails to provide certified intervals for influence under data shifts [2512.09103].

## 2. Wasserstein Uncertainty Sets and Rationale

The $p$-Wasserstein distance between distributions $P$ and $Q$ on $\mathcal{Z}$ is
\[
W_p(P,Q) = \inf_{\pi\in\Pi(P,Q)} \left(\mathbb{E}_{(z,z')\sim\pi}\left[\|z-z'\|^p\right]\right)^{1/p}
\]
The corresponding Wasserstein ball $\mathcal{B}_p(P,\epsilon) = \{Q : W_p(Q,P) \le \epsilon\}$ defines an adversarial uncertainty set for robust analysis. Wasserstein metrics are preferred over Euclidean parameter perturbations because they quantify distributional (mass transport) shifts, accommodate support changes such as outlier addition/removal, and allow tractable duality-based reformulations via Kantorovich–Rubinstein duality. This construction provides a natural notion of uncertainty for robust influence function analysis [2512.09103].

## 3. Definition and Computation of W-RIF

For any $x\in\mathcal{Z}$, the W-RIF at radius $\epsilon$ is
\[
\mathrm{W\text{-}RIF}(x; \theta, P, \epsilon) = \sup_{Q\in\mathcal{B}_p(P, \epsilon)} 
\left| \mathbb{E}_{z\sim Q} \left[\nabla_\theta L(\theta;z)^\top \nabla_\theta L(\theta; x)\right] \right|
\]
Substituting empirical estimators $\theta = \hat{\theta}$ and $P = P_n$, a first-order expansion yields
\[
\mathcal{I}_Q = \mathcal{I}_{P_n} + \int_{\mathcal{Z}} S(z) [dQ(z) - dP_n(z)] + O(\|Q - P_n\|^2)
\]
where the complete sensitivity kernel $S(z)$ is the sum of $S_H(z)$ and $S_g(z)$, both involving Hessian and gradient terms. If $S$ is $L_S$-Lipschitz with respect to the input norm, the dual form implies
\[
\sup_{Q:W_1(Q,P_n)\le \epsilon} \int S(z)\, (dQ - dP_n) = \epsilon L_S
\]
leading to the closed-form certified interval
\[
\mathcal{I}(z_i, z_{\text{test}}) \pm \epsilon L_S + O(\epsilon^2)
\]
This interval is guaranteed to contain the leave-one-out or true population-level influence as detailed below [2512.09103].

## 4. Provable Certification and Coverage Guarantees

Leave-one-out influence, given by removing $z_i$ from $P_n$, is a specific distributional perturbation: $W_1(P_n, P_{n,-i}) \le \frac{\text{diam}(\mathcal{Z})}{n}$. Setting $\epsilon \ge \text{diam}(\mathcal{Z})/n$, the W-RIF interval around $P_n$ certifies the true leave-one-out influence. For population-level guarantees, one selects
\[
\epsilon_n = O(n^{-1/2})
\]
so that, with probability at least $1-\delta$, $W_p(P_n, P) \le \epsilon_n$ and the population influence is covered:
\[
\Pr\left[ \mathcal{I}_{P}(x) \in \mathcal{I}_{P_n}(x) \pm \epsilon_n L_S \right] \ge 1-\delta
\]
This provides formal certification for robust attribution and data influence quantification in convex models [2512.09103].

## 5. Computational Strategies

Exact estimation of $L_S$ by pairwise slope calculation has computational complexity $O(n^2p^2 + p^3)$. Practical approximations include:
- Estimating $\|\nabla_z S(z)\|$ at a random subset of points.
- Solving the dual optimization
  \[
  \max_{\phi:\,\mathrm{Lip}(\phi)\le 1}\;\frac{1}{n}\sum_{i=1}^n S(z_i)\phi(z_i)
  \]
  using projected gradient methods, which scale linearly in $n$.
All other computations (Hessian inversion, gradient evaluation) adhere to standard influence-function workflows [2512.09103].

## 6. Extensions to Deep Networks and the Spectral Amplification Barrier

In non-convex deep networks, the parameter solution map $\hat{\theta}(Q)$ may change discontinuously under data perturbations, rendering classical W-RIF constructions invalid. TRAK (a linearized attribution method at the fixed network) is formulated as
\[
\mathrm{TRAK}(z_{\text{test}}, z_i) = \phi(z_{\text{test}})^\top Q^{-1} \phi(z_i),\quad Q = \frac{1}{n}\sum_j \phi(z_j)\phi(z_j)^\top + \lambda I
\]
where $\phi(z) = \nabla_\theta f_{\hat{\theta}}(z)$. However, naive use of Euclidean $W_2$-balls in feature space results in vacuous certificates, because the relevant Lipschitz constant scales inversely with the smallest eigenvalue of $Q$, and deep representations typically exhibit ill-conditioning with condition numbers of $10^4$–$10^6$. Empirically, such Euclidean certificates cover $0\%$ of ranking pairs [2512.09103].

## 7. Natural Wasserstein Metric and Robust Neural Attribution

To address spectral amplification, the Natural Wasserstein metric is defined as
\[
d_{\mathrm{Nat}}(z, z') = \|\phi(z) - \phi(z')\|_{Q^{-1}} = \sqrt{(\phi(z) - \phi(z'))^\top Q^{-1} (\phi(z) - \phi(z'))}
\]
In this induced geometry, the Lipschitz constant of the attribution score coincides with the data-dependent “Self-Influence” score:
\[
\mathrm{SI}(z) = \phi(z)^\top Q^{-1} \phi(z) = \|Q^{-1/2}\phi(z)\|_2^2
\]
The critical bound is
\[
L_{\mathrm{Nat}}\le 2\,\sqrt{\mathrm{SI}(z_{\text{test}})}\sqrt{\mathrm{SI}(z_i)}\max_{j}\sqrt{\mathrm{SI}(z_j)}
\]
Empirically, this yields certified intervals that are $10$–$100\times$ tighter than Euclidean baselines. On CIFAR-10 with ResNet-18, Natural W-TRAK certificates cover $68.7\%$ of ranking pairs, in contrast to $0\%$ for Euclidean approaches [2512.09103].

Furthermore, Self-Influence not only certifies attribution robustness but also provides a mathematically grounded leverage score for anomaly detection, achieving AUROC $0.970$ for label noise detection and identifying $94.1\%$ of corrupted labels when considering the top $20\%$ of the training data [2512.09103].

---

Wasserstein-Robust Influence Functions unify classical and modern approaches to data attribution by extending influence function analysis to robust, distributional settings. For convex models, W-RIF yields $O(\epsilon^2)$-accurate certified intervals with provable guarantees for leave-one-out and population influence. For deep networks, robust certification is only achievable by linearizing at the feature level and measuring perturbations in the Natural Wasserstein metric, thereby circumventing the issue of spectral amplification. The theory provides a formal basis for certified data valuation, debugging, unlearning, and robust anomaly detection in high-dimensional, non-convex machine learning models [2512.09103].

Source: https://www.emergentmind.com/topics/wasserstein-robust-influence-functions-w-rif