Differentially Private Auditing Under Strategic Response

Abstract: Regulatory audits of AI systems increasingly rely on differential privacy (DP) to protect training data and model internals. We study audit design when the audited developer can strategically respond to the privacy-constrained audit interface. We formalize privacy-constrained auditing as a bilevel Stackelberg game, in which an auditor commits to a query policy and DP budget allocation across harm dimensions, and a strategic developer reallocates mitigation efforts in response. We introduce the welfare-weighted under-detection gap $B_w$, the welfare-weighted true residual harm the audit fails to detect at the developer's strategic best response, and prove that naive DP auditing (uniform or harm-proportional allocation) induces a strictly larger $B_w$ than any non-strategic mitigation baseline whenever effective detectability is heterogeneous, the welfare weights are not comonotone with detectability, and the developer's optimum is interior. We characterize the optimal auditor allocation as a four-factor balance of welfare weight, audit miss-probability, detectability elasticity, and mitigation-cost curvature, and provide a single-level reformulation of the bilevel problem via the developer's KKT system. We propose Strategic Private Audit Design (SPAD), a projected-gradient algorithm with hypergradients computed through the developer's best response.

• The paper formalizes DP auditing as a bilevel Stackelberg game in which developers redirect mitigation toward highly detectable harms, causing naive uniform and harm-proportional policies to miss more welfare-weighted harm.
• The paper derives an optimal allocation rule balancing welfare weights, miss probability, privacy detectability elasticity, and mitigation-cost curvature, and reformulates the problem as an MPCC solved approximately by the SPAD algorithm.
• The paper’s synthetic experiments show SPAD reduces the under-detection gap by 5–14% under uniform welfare and up to 44% with heterogeneous welfare weights, while relying on assumptions that require real-world validation.

Overview

This paper studies a structural weakness in differentially private (DP) auditing of AI systems: when an auditor commits to a query policy and privacy-budget allocation across harm dimensions, a strategic developer can reallocate mitigation effort toward dimensions where DP noise and sparse querying make harm least detectable. The authors formalize this as a bilevel Stackelberg game between an auditor (leader) who chooses per-dimension query rates $\pi$ and DP budgets $\varepsilon$, and a developer (follower) who minimizes expected audit exposure plus mitigation cost rather than true welfare-weighted residual harm. The central object of audit quality is the welfare-weighted under-detection gap $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$, which measures welfare-weighted residual harm that escapes detection at the developer's best response.

Model

The setting has $d$ harm dimensions (e.g., demographic subgroups or failure modes) with baseline harms $h$, welfare weights $w$, and residual-harm functions $g_j(h_j, m_j)$ decreasing in mitigation $m_j$. The auditor's interface returns DP-noised observations whose effective detectability is captured by a reduced-form function $\alpha_j(\varepsilon_j)$: strictly increasing, continuously differentiable, with $\alpha_j(0)=0$ and $\varepsilon$0. The developer solves a convex program minimizing $\varepsilon$1 subject to a cost budget $\varepsilon$2; strict convexity and coercivity of costs guarantee a unique interior best response characterized by the first-order condition $\varepsilon$3. The key asymmetry is that the developer equalizes marginal exposure reduction per cost across dimensions, so high-detectability dimensions receive disproportionate mitigation while low-detectability dimensions accumulate residual harm.

A notable modelling abstraction is that detectability depends on $\varepsilon$4 alone rather than jointly on $\varepsilon$5. The appendix shows that canonical mechanisms (Gaussian, Laplace, randomised response) yield joint detectability $\varepsilon$6, and that equilibrium residual harm deviates from the fixed reference point by $\varepsilon$7 (median deviation ≈ 1.07) — so the reduced form is a tractability choice, not a small perturbation. Qualitative conclusions survive; quantitative allocations change.

Strategic blind spots under naive auditing

The main negative result (Theorem 1) states that naive allocations — uniform or harm-proportional — induce a strictly larger $\varepsilon$8 at the developer's strategic best response than at any budget-exhausting non-strategic baseline, provided detectability is heterogeneous, welfare weights are not comonotone with detectability, the optimum is interior, and residual harm is anti-monotone in detectability. The proof decomposes the gap difference into a $\varepsilon$9-weighted rearrangement sum and applies a strict rearrangement inequality.

The anti-monotonicity hypothesis (vi) deserves scrutiny: it holds exactly only under homogeneous parameters, and the paper concedes it is violated in roughly half of pairs under heterogeneous parameters. Empirically, however, the theorem's conclusion held in 1600/1600 verified cells for both baselines, suggesting a weaker net-rearrangement condition suffices — which the authors leave as an open characterization. Hypothesis (iv) also fails on a measure-zero set where $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$0 and $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$1 are comonotone, collapsing the inequality to equality.

A corollary establishes that harm-proportional allocation is not merely suboptimal but not even locally optimal: shifting budget from the lowest-elasticity to the highest-elasticity dimension strictly decreases $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$2 via both direct coverage effects and indirect developer-response effects. A worked three-dimensional example quantifies the mechanism: moving from uniform to welfare-aware allocation reduces $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$3 by about 17% despite residual harm migrating toward low-detectability dimensions.

Optimal audit design

Three results characterize the optimal policy. First, the optimal allocation is generically non-harm-proportional (Theorem 2): the auditor's stationarity condition balances four factors — welfare weight, miss-probability, detectability elasticity $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$4, and an effective mitigation-cost curvature aggregate $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$5 combining harm and cost curvature. Cheap-to-mitigate, high-elasticity, high-welfare dimensions warrant larger budgets. Second, the bilevel problem admits a single-level MPCC reformulation via the developer's KKT conditions (Theorem 3), valid under MPEC-LICQ and strict complementarity — generic but not automatic assumptions; if the developer's budget constraint is slack ($$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$6), gradient methods may converge to W-stationary points that are not KKT points of the original problem. Third, the SPAD algorithm computes near-optimal allocations via projected gradient ascent with hypergradients through the developer's best response (implicit differentiation or $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$7 finite differences), at complexity $$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$8–$$B_w(\pi,\varepsilon) = \sum_j w_j (1 - \pi_j\alpha_j(\varepsilon_j)) g_j(h_j, m^*_j)$$9.

For tight budgets, Proposition 1 gives a lower bound at $d$0: for $d$1, no feasible audit achieves $d$2 as $d$3 — some dimension necessarily goes unmitigated, and the auditor optimally sacrifices the smaller-$d$4 one. The general-$d$5 extension remains a conjecture requiring a uniform bound on detectability concentration, open under Assumption 1 alone. A robust variant handles uncertainty over developer types via sample average approximation or alternating min–max iterates, though convergence guarantees are deferred.

Evaluation

The synthetic protocol spans $d$6, exponential detectability with heterogeneous $d$7, quadratic and power-law costs, $d$8, and three developer types (fully strategic, boundedly rational, non-strategic). The reported evidence is a single preliminary ablation (8,000 evaluations, 50 seeds each): under uniform welfare, SPAD reduces $d$9 by 5–14% versus uniform and 5–12% versus harm-proportional allocation; under Dirichlet welfare weights, reductions reach 20–44% versus uniform, 20–41% versus harm-proportional, and 13–20% versus welfare-proportional — the last gap isolating the value of strategic-awareness beyond welfare-targeted budgeting. All gaps exceed bootstrap 95% CIs by an order of magnitude. At $h$0 the lower bound binds all baselines equally, and boundedly rational developers shrink magnitudes without reversing signs. These numbers rest entirely on quadratic costs, exponential residual harm, dense harm priors, and simulated developers; whether real developers behave strategically under DP-audited interfaces is explicitly left as an empirical question for a companion paper.

Limitations and open questions

The framework assumes one-shot commitment, exact developer best responses, and mitigation-only adaptation; relaxations such as repeated audits, noisy knowledge of $h$1, concealment strategies, or bounded rationality preserve Theorem 1 qualitatively but not SPAD's optimality claims. The reduced-form detectability $h$2 ignores the joint dependence on equilibrium residual harm, with approximation errors that are not pointwise small. Hypothesis (vi)'s weaker net-rearrangement replacement, the general-$h$3 lower bound, convergence guarantees for the robust min–max variant, and empirical validation against actual developer behavior all remain open. The broader-impacts discussion flags two practical concerns: SPAD requires knowledge of the developer's cost structure and budget, and a misaligned auditor could manipulate welfare weights $h$4 to game the design — motivating independent certification of $h$5 and simplified threshold-based rules.

Conclusion

The paper identifies and formalizes a previously unmodeled triple interaction — DP composition across harm dimensions, auditor commitment, and strategic mitigation reallocation — and shows that conventional naive DP audit designs systematically enlarge undetected welfare-weighted harm. Its contributions are a clean negative result for uniform and harm-proportional policies, a four-factor characterization of optimal allocation, a tractable KKT-based reformulation with the SPAD algorithm, and preliminary synthetic evidence of double-digit gap reductions under non-uniform welfare. The theoretical core is solid within its stated assumptions, though the reliance on a reduced-form detectability model, a sufficient-condition proof step violated in many heterogeneous instances, and purely synthetic evaluation mean the quantitative claims await broader validation.