Inverse Hessian Attack (IHA)

• Inverse Hessian Attack (IHA) is a white-box membership inference technique that exploits SGD dynamics and inverse-Hessian vector products to determine if a record was used in training.
• It derives an optimal membership score by combining loss values with second-order terms under the assumption of a locally quadratic loss surface near minima.
• IHA requires full model access and computationally intensive Hessian inversion, making it more suitable for high-information privacy audits rather than routine threat assessments.

Searching arXiv for the cited IHA paper and closely related work on iHVPs, membership inference, and transfer-learning evaluations. Inverse Hessian Attack (IHA) is a white-box membership inference attack for stochastic gradient descent (SGD)-trained models that explicitly uses model parameters, curvature, and inverse-Hessian vector products (iHVPs) to infer whether a record was used in training (Suri et al., 2024). In contrast to black-box attacks that rely primarily on the loss or model outputs, IHA is derived from discrete-time SGD dynamics under assumptions that the loss surface is approximately quadratic near local minima, the training dynamics reach a stationary state, and the Hessian and loss value at the optimum change little by removing a single training data point (Suri et al., 2024). The method is presented as evidence that white-box access can reveal more about membership than loss-based black-box attacks in common practical settings, while also functioning as a privacy auditing tool whose applicability is constrained by computational cost and unusually strong data-access assumptions (Suri et al., 2024).

1. Origin and threat model

IHA was introduced in "Do Parameters Reveal More than Loss for Membership Inference?" (Suri et al., 2024). The paper studies membership inference attacks (MIAs), which try to determine if a specific data point was present in a model's training set, and argues that prior claims around black-box access being sufficient for optimal membership inference do not hold for stochastic gradient descent (Suri et al., 2024). In the formulation given there, optimal membership inference indeed requires white-box access, and this motivates IHA as an explicit white-box inference attack that takes advantage of computing inverse-Hessian vector products (Suri et al., 2024).

The attack assumes access to the final trained parameters of the target model, knowledge of training hyperparameters, and knowledge of all training records except possibly the target record (Suri et al., 2024). The transfer-learning evaluation later characterizes this as breaking the “standard MIA threat model,” since standard attacks typically do not assume access to model weights together with all but the target record in the training set (Bai et al., 7 Oct 2025). The same evaluation states that IHA requires access to all but the target record in a training set and the final trained parameters of the target model, and describes the attack as a white-box membership inference attack specifically designed to leverage deeper access to a target model compared to standard MIAs (Bai et al., 7 Oct 2025).

This threat model makes IHA realistic for a privacy auditor, but not directly feasible for a real adversary in the ordinary sense described in the source material (Suri et al., 2024). A plausible implication is that IHA is best interpreted as a high-information or worst-case audit mechanism rather than as a baseline for routine black-box privacy risk estimation.

2. Theoretical derivation from SGD dynamics

The derivation of IHA in (Suri et al., 2024) departs from earlier theory that modeled the parameter distribution after training with a Boltzmann distribution,

$$\mathbb{P}(\mathbf{w} | \text{train data}) \propto \exp\left(-\frac{1}{T} \sum_i m_i \ell(\mathbf{w}, \mathbf{z}_i)\right),$$

where $m_i$ indicates membership and $\ell$ is the loss. The paper argues that these assumptions do not hold for typical, finite-step stochastic gradient descent training used in practice (Suri et al., 2024).

Instead, the authors leverage recent advances in the theoretical understanding of discrete-time SGD dynamics and make three assumptions: the loss surface is approximately quadratic near local minima, the training dynamics reach a stationary state, and the Hessian and loss value at the optimum change little by removing a single training data point (Suri et al., 2024). Under these assumptions, the paper derives an explicit formula for the optimal membership inference score when white-box access and knowledge of training hyperparameters are available (Suri et al., 2024).

The resulting score, presented in the source as Equation 7, is

$$\text{IHA}(\mathbf{z}_1) := \frac{\ell(\mathbf{w}, \mathbf{z}_1)}{1 + \mu} - \frac{1}{\lambda}\left( \frac{1}{n} \| \mathbf{H}_*^{-1} \nabla \ell(\mathbf{w}, \mathbf{z}_1) \|^2 + 2 (\mathbf{H}_*^{-1} \nabla L_0(\mathbf{w}))^\top (\mathbf{H}_*^{-1}\nabla \ell(\mathbf{w}, \mathbf{z}_1)) \right).$$

Here, the quantities identified in the source are $\mathbf{H}_*$ as the Hessian at the local minimum, $\ell(\mathbf{w}, \mathbf{z}_1)$ as the loss on the record in question, $\nabla \ell(\mathbf{w}, \mathbf{z}_1)$ as the gradient with respect to the model parameters, $L_0(\mathbf{w})$ as the mean loss excluding $\mathbf{z}_1$, $n$ as the total number of training records, $m_i$0 as the learning rate, $m_i$1 as momentum, and $m_i$2 as batch size (Suri et al., 2024).

The transfer-learning evaluation states the mathematical intuition differently, emphasizing a local similarity assumption: models trained with or without a single data point converge to nearby local minima, and thus the Hessian matrices at these minima are expected to be similar (Bai et al., 7 Oct 2025). It summarizes this as

$m_i$3

with $m_i$4 and $m_i$5 denoting parameters with and without the target sample (Bai et al., 7 Oct 2025). This suggests that the attack’s second-order terms are intended to encode the sensitivity of the learned optimum to inclusion or exclusion of an individual record.

3. Structure of the score and relation to prior attacks

The source material explicitly positions IHA relative to several prior attack families (Suri et al., 2024). A traditional loss attack uses only the loss, that is, $m_i$6 (Suri et al., 2024). Reference or calibration attacks such as those associated with Sablayrolles et al. and Carlini et al. use multiple reference models to calibrate the loss or perform likelihood-ratio tests (Suri et al., 2024). Self-influence uses $m_i$7 as a membership signal and is described as closely related to a term in the IHA formula (Suri et al., 2024).

Within this comparison, IHA is described as a more principled, theoretically optimal score under SGD-based training because it combines the loss and second-order or statistical terms reflecting the expected model change from removing the queried record (Suri et al., 2024). The use of the inverse Hessian is stated to capture how a data point influences the entire parameter space, correcting for “easier” or “harder” points (Suri et al., 2024). That framing links IHA to a broader line of research on inverse Hessian-vector products and influence-style reasoning, even though the membership objective differs from training data attribution.

A closely related strand of work revisits iHVP computation for influence functions and explains the role of the inverse Hessian as reweighting gradient directions by curvature, de-emphasizing directions with high curvature and amplifying low-curvature directions (Klochkov et al., 2024). That paper gives the influence formula

$m_i$8

and argues that the inverse Hessian acts to "whiten" gradient directions (Klochkov et al., 2024). Although this is not itself IHA, it clarifies why the second-order terms in IHA are not reducible to loss alone.

4. Practical implementation and computational bottlenecks

The practical implementation described in (Suri et al., 2024) requires computing two iHVPs: $m_i$9 In the experiments reported there, on small models such as linear models and 2-layer nets, the authors compute the full Hessian and its inverse directly, with damping for near-zero or negative eigenvalues to address ill-conditioning (Suri et al., 2024). For larger models, the paper states that this would not scale, and mentions techniques like LiSSA or conjugate gradient as potential but error-prone alternatives for approximate iHVPs (Suri et al., 2024).

The same paper also notes that no reference models are needed, unlike strong black-box attacks, and that the score is used directly as the membership score, with higher values indicating that a point is more likely to be a member (Suri et al., 2024). However, the need to compute $\ell$0 by summing the gradients over all other training data further increases the attack’s operational burden (Suri et al., 2024).

Subsequent work on iHVPs provides additional context for this bottleneck. "Revisiting inverse Hessian vector products for calculating influence functions" presents LiSSA as a stochastic iterative approach for approximating iHVPs, with recursion

$\ell$1

and argues that its hyperparameters can be selected using spectral properties of the Hessian, particularly the trace and largest eigenvalue (Klochkov et al., 2024). More recently, "Better Training Data Attribution via Better Inverse Hessian-Vector Products" introduces ASTRA, which uses the EKFAC-preconditioner on Neumann series iterations to obtain accurate iHVP approximations for training data attribution (Wang et al., 19 Jul 2025). ASTRA is described as easy to tune, requiring fewer iterations than Neumann series iterations, and being more accurate than EKFAC-based approximations (Wang et al., 19 Jul 2025). The source explicitly notes interpretability and security implications, including that more accurate iHVPs can make it easier to identify training points that, if manipulated, have maximal or targeted influence on model predictions, while also enabling more robust detection of malicious or noisy training data (Wang et al., 19 Jul 2025).

This suggests that the computational limitations identified in the original IHA work are not fixed properties of the attack itself; rather, they are tightly coupled to the state of iHVP approximation methods.

5. Empirical findings

The original empirical evaluation in (Suri et al., 2024) tests IHA on Purchase-100, MNIST-odd/even, and FashionMNIST. The metrics reported are attack AUC and TPR at low FPR, specifically at $\ell$2 and $\ell$3 false positive rate (Suri et al., 2024). According to the source, IHA outperforms all baselines on Purchase-100 and FashionMNIST, and on Purchase-100 the reported AUC values are .709 for IHA, .645 for LiRA, and .531 for LOSS (Suri et al., 2024). The same source also states that IHA does not dominate in every setting, citing MNIST-odd as an example, but always achieves at least competitive or better performance (Suri et al., 2024).

A later empirical comparison in deep transfer learning extends the evaluation regime and states that there is no one MIA which captures all privacy risks in models trained with transfer learning (Bai et al., 7 Oct 2025). In that study, LiRA demonstrates superior performance across most experimental scenarios, while IHA proves to be more effective against models fine-tuned on PatchCamelyon in high data regime (Bai et al., 7 Oct 2025). The paper evaluates CIFAR-10, CIFAR-100, and PatchCamelyon using ViT-B/16 and BiT-M-R50x1 backbones pre-trained on ImageNet-21k, with head-only, ALL, and FiLM parameterization modes, and averages results over 10 experimental repeats (Bai et al., 7 Oct 2025).

The transfer-learning paper highlights a specific asymmetry across data regimes. On PatchCamelyon with high shots, IHA’s efficacy remains stable or increases as data scale grows, whereas standard MIAs typically decrease in efficacy (Bai et al., 7 Oct 2025). It further reports that for PatchCamelyon, ViT-B/16, head-only, $\ell$4, LiRA has TPR@FPR$\ell$5, while IHA has TPR@FPR$\ell$6 in the range $\ell$7–$\ell$8 (Bai et al., 7 Oct 2025). By contrast, on CIFAR-10 at similar scales, IHA does not excel in the same way, and its advantage is described as most pronounced in PatchCamelyon (Bai et al., 7 Oct 2025).

Setting	Reported outcome	Source
Purchase-100	IHA AUC .709 vs. .645 LiRA, .531 LOSS	(Suri et al., 2024)
FashionMNIST	IHA outperforms all baselines	(Suri et al., 2024)
MNIST-odd/even	IHA does not dominate in every setting, but is at least competitive or better	(Suri et al., 2024)
PatchCamelyon, high data regime	IHA more effective than LiRA in reported low-FPR evaluations	(Bai et al., 7 Oct 2025)

Taken together, these results support a narrow but important conclusion: IHA is not uniformly strongest across all tasks, but it can expose privacy leakage that score-based MIAs do not capture, especially when its stronger threat model is plausible (Bai et al., 7 Oct 2025).

6. Limitations, scope, and controversies

The limitations of IHA are explicit in the source material. The original paper states that the attack requires full access to model parameters, all-but-one training data, and training hyperparameters (Suri et al., 2024). It also requires computing the full Hessian and its inverse in the reported experiments, which is computationally expensive and only tractable for small models in practice (Suri et al., 2024). The theory assumes a locally quadratic loss surface and that removing one record does not significantly affect the Hessian or minimum; these assumptions are described as approximately valid for large datasets and relatively smooth models, but less so for highly overparameterized or hardly regularized ones (Suri et al., 2024). The attack also assumes that SGD stationary-state conditions are met and that model training follows the usual SGD dynamics, so unusual optimizers or aggressive learning rate schedules may invalidate the model (Suri et al., 2024).

The transfer-learning comparison reiterates that IHA’s required knowledge is rarely realistic outside small-scale or managed or insider threat settings (Bai et al., 7 Oct 2025). It also notes fluctuating performance with increasing dataset size, possibly due to instability in Hessian approximations and sensitivity to model initialization or optimization (Bai et al., 7 Oct 2025). Because of this, the paper advises combining black-box methods such as LiRA with white-box methods such as IHA for practical auditing (Bai et al., 7 Oct 2025).

A recurrent misconception in the surrounding literature is that black-box access is sufficient for optimal membership inference. The central contribution of (Suri et al., 2024) is precisely to dispute that claim for SGD-trained models. Another potential source of confusion is terminological: a separate adversarial-learning paper uses the phrase “IHA framework” to refer to inverse Hessian-based adversarial perturbations of the form

$\ell$9

in the context of second-order adversarial examples (Qian et al., 2022). That usage concerns input-space adversarial example generation rather than membership inference, even though both rely on inverse-Hessian structure.

7. Relation to inverse-Hessian methods beyond membership inference

IHA belongs to a wider methodological family in which inverse Hessians or iHVPs are used to characterize sensitivity. In influence functions and training data attribution, iHVPs arise when estimating how changes to a training sample affect a function of a test point’s prediction (Wang et al., 19 Jul 2025). The source gives the influence-function quantity

$$\text{IHA}(\mathbf{z}_1) := \frac{\ell(\mathbf{w}, \mathbf{z}_1)}{1 + \mu} - \frac{1}{\lambda}\left( \frac{1}{n} \| \mathbf{H}_*^{-1} \nabla \ell(\mathbf{w}, \mathbf{z}_1) \|^2 + 2 (\mathbf{H}_*^{-1} \nabla L_0(\mathbf{w}))^\top (\mathbf{H}_*^{-1}\nabla \ell(\mathbf{w}, \mathbf{z}_1)) \right).$$0

and emphasizes that direct computation is infeasible for large models, motivating approximate solvers such as Neumann-series methods, EKFAC, LiSSA, and ASTRA (Wang et al., 19 Jul 2025).

The relevance of this broader literature to IHA is twofold. First, IHA’s feasibility is directly determined by how well iHVPs can be approximated under realistic compute budgets (Suri et al., 2024). Second, the security implications of improved iHVP computation are dual-use: more accurate second-order analysis can improve auditing and interpretability, but may also facilitate stronger attacks or more targeted manipulation of training data (Wang et al., 19 Jul 2025). A plausible implication is that advances in scalable iHVP approximation may shift IHA from a small-model auditing technique toward a more general white-box privacy analysis tool, provided the strong access assumptions remain acceptable for the intended evaluation setting.