---
title: Privacy Odometers in Differential Privacy
url: https://www.emergentmind.com/topics/privacy-odometers
type: topic
---

# Privacy Odometers in Differential Privacy

A privacy odometer is a formal mechanism within the theory of differential privacy that provides a time-uniform, high-probability upper bound on the cumulative privacy loss incurred by a sequence of (potentially adaptively chosen) differentially private mechanisms. Unlike privacy filters, which enforce a pre-specified privacy budget via a stopping rule, privacy odometers allow analysts to track privacy expenditure dynamically without the need to fix a budget in advance. This is critical for practical deployments where the number of data accesses and their privacy parameters may depend on previous computations' outcomes, settings where classical (fixed-parameter) composition theorems fall short. Privacy odometers provide mathematically rigorous, high-probability guarantees that, at every time step, the true cumulative privacy loss does not exceed the odometer's reading, except with a specified small failure probability.

## 1. Formal Definitions and Guarantee

A privacy odometer is defined relative to a target failure probability $\delta \in (0,1)$, a sequence of mechanisms $A_1, A_2, \ldots$ (each $(\epsilon_n, \delta_n)$-differentially private, possibly chosen adaptively based on previous outcomes), and neighboring databases $x \sim x'$. At each round $n$, the privacy loss random variable is
\[
L_{1:n}(x, x') = \sum_{m=1}^n L_m(x, x'), \quad \text{where} \quad L_m(x, x') = \ln\frac{p_{m}(A_m(x)\mid A_{1:m-1}(x))}{p_{m}(A_m(x)\mid A_{1:m-1}(x'))}.
\]
A family of deterministic functions $u_n$ defines the odometer readings $U_n(x) = u_n(\epsilon_1(x), ..., \epsilon_{n-1}(x); \delta_1(x), ..., \delta_{n-1}(x))$. The sequence $(u_n)_{n \geq 1}$ is a $\delta$-privacy odometer if, for every $x \sim x'$,
\[
\Pr\left[ \exists n \geq 1 : L_{1:n}(x, x') > U_n(x) \right] \leq \delta.
\]
That is, with probability at least $1-\delta$, the true privacy loss never exceeds the instantaneous odometer reading at any step [1605.08294], [2203.05481].

## 2. Odometer Bounds and Constructions

Recent work, notably Whitehouse et al. [2203.05481], provides several odometer constructions with varying tightness of their bounds and dependencies on the adaptivity of parameters. Assuming each mechanism $A_n$ is $(\epsilon_n,\delta_n)$-pointwise DP conditioned on the past, they define $V_n = \sum_{m=1}^n \epsilon_m^2$. For a total allowable failure $\delta = \delta' + \delta''$, principal odometer forms are:

- **Filter odometer:** Matches the classical advanced composition bound at a chosen $V_n = y^*$. The guarantee is
  \[
  \Pr\left\{ \exists n: L_{1:n} > u_n^F \right\} \leq \delta,
  \]
  where $u_n^F$ provides the same leading constants as advanced composition [2203.05481].

- **Mixture odometer:** Valid for all $n$ simultaneously, providing a bound of order $\sqrt{2 V_n \ln V_n}$, derived via the method of mixtures for martingale concentration.

- **Stitched odometer:** Achieves the optimal asymptotic rate $O(\sqrt{V_n \ln\ln V_n})$ via a stitching argument, which closely matches the law of the iterated logarithm for sums of bounded random variables.

All above guarantee, simultaneously for all $n$, that the realized privacy loss stays beneath the associated odometer with probability $1-\delta$.

A foundational result in [1605.08294] demonstrates that, in the adaptive-parameter setting, privacy odometers must pay an extra factor of $\sqrt{\ln\ln n}$ in the cumulative loss comparison to filters or nonadaptive advanced composition. Thus, odometers provide nearly tight adaptive composition, but with an unavoidable, though slowly growing, overhead.

## 3. Martingale-Based Analysis and Proof Techniques

The derivation of privacy odometers is grounded in martingale concentration inequalities. The privacy-loss martingale is
\[
M_n = \sum_{m=1}^n \left( L_m(x, x') - \mathbb{E}\left[ L_m(x, x') \mid \mathcal{F}_{m-1} \right] \right),
\]
with the filtration $\mathcal{F}_n$ determined by the history of $A_1, ..., A_n$.

Key technical steps [2203.05481], [1605.08294]:

- The one-step conditional expectation is bounded as $\mathbb{E}\left[ L_m(x, x') \mid \mathcal{F}_{m-1} \right] \leq \frac{1}{2} \epsilon_m^2$.
- Uniform (over time) concentration of the privacy-loss martingale is obtained by advanced tools:
  - **Line-crossing bounds:** Provide explicit expressions for the maximal excursion of the martingale, yielding the filter odometer.
  - **Mixture and stitching bounds:** Further refine tail decay at the expense of iterated logarithmic factors in the bound, enabling tight, time-uniform guarantees.

These techniques replace earlier combinatorial or moment-based arguments and allow for sharp, implementation-friendly bounds when privacy parameters are chosen adaptively.

## 4. Advanced Composition and Performance Comparison

The classical advanced composition result states that composing $n$ $(\epsilon_m,0)$-DP mechanisms achieves $(\epsilon, \delta')$-DP with
\[
\epsilon = \sqrt{2 \ln(1/\delta') \sum_{m=1}^n \epsilon_m^2} + \sum_{m=1}^n \frac{\epsilon_m (e^{\epsilon_m} - 1)}{e^{\epsilon_m} + 1}.
\]
For small $\epsilon_m$, the second term is $O(\sum \epsilon_m^2)$. In the nonadaptive case, privacy filters or filters can match this bound exactly (up to constants), while privacy odometers, which cannot "refuse" output, must suffer the $\sqrt{\ln\ln n}$ overhead. This formal separation is proved in [1605.08294], affirming that any valid odometer in this setting must incur this additional term, while filters or nonadaptive composition can avoid it.

Empirical evaluation in deep learning [2103.01379] shows that RDP-based odometers with improved constants report realized $\epsilon$ up to $2$–$10\times$ lower than previous constructions, making them practical for adaptive algorithm design.

## 5. Rényi Differential Privacy Odometers

The odometer concept readily extends to Rényi Differential Privacy (RDP) [2103.01379], which offers additive composition for orders $\alpha > 1$. For $(\alpha, \epsilon_i)$-RDP mechanisms, composing adaptively chosen steps summing to $\sum_i \epsilon_i \leq \mathcal{B}$ results in $(\alpha, \mathcal{B})$-RDP.

In this setting, the RDP odometer maintains a collection of exponentially growing privacy budgets indexed by $f$, with each "bucket" $\rho^f(\alpha)$ of the form $2^{f-1} L(\alpha)$ where $L(\alpha) = \log(2|\Lambda|/\delta)/(\alpha - 1)$. The odometer reading after $k$ steps is
\[
\epsilon_{\mathrm{odo}} = \min_{\alpha \in \Lambda} \Bigg\{ \rho^f(\alpha) + \frac{\log(2 f^2 |\Lambda|/\delta)}{\alpha - 1} \Bigg\},
\]
and, as in the classical setting, this provides a high-probability upper bound, accounting for the adaptivity in the privacy parameters.

Notably, the RDP odometer incurs only an additive, logarithmic-in-$f$ penalty compared to the strong-composition rate, with no adaptivity overhead in the filter version.

## 6. Practical Implementation, Use Cases, and Experimental Insights

Practical deployment of privacy odometers relies on their ability to dynamically report privacy usage, making them suitable for scenarios such as adaptive noise or batch-size tuning during deep learning training, and early-stopping algorithms to conserve the privacy budget [2103.01379], [2203.05481].

A typical implementation maintains a summary of the cumulant privacy loss statistics (e.g., $\sum \epsilon_j^2$ for DP or $\sum \epsilon_j(\alpha)$ for RDP), updating the odometer reading efficiently using bucketed structures for high-frequency queries.

Experiments on DP-SGD for CIFAR-10 classification demonstrate that privacy odometers enable:

- Adaptive algorithm design (e.g., dynamically tuning noise or batch size for improved accuracy at fixed $\epsilon$);
- Early halting, saving privacy budget when further training yields diminishing returns;
- Substantially tighter $\epsilon$-accounting relative to previous adaptive tools, translating into greater utility for given privacy standards.

## 7. Fundamental Limitations and Theoretical Separation from Filters

A principal limitation, established by Rogers et al. [1605.08294], is that privacy odometers must suffer an additional $\sqrt{\ln\ln n}$ factor compared to privacy filters or nonadaptive advanced composition. This gap is intrinsic and arises from the impossibility (proved via an anti-concentration argument) of producing an odometer bound that uniformly matches the advanced composition rate for all adversarial stopping-time strategies. By contrast, privacy filters can always enforce the advanced composition budget (up to constants) by halting future computations once the cumulative loss meets the enforced threshold.

This theoretical separation underlines the necessity of choosing between the flexibility of pay-as-you-go privacy accounting (odometers) and the tightness of fixed-budget accounting (filters), with the former optimal for interactive, open-ended analyses and the latter for pre-specified query regimes.

---

**References**
- Rogers, Roth, Smith, Thakurta. "Privacy Odometers and Filters: Pay-as-you-Go Composition" [1605.08294]
- Whitehouse, Steinke, Bun, et al. "Fully Adaptive Composition in Differential Privacy" [2203.05481]
- Feldman, Zrnic. "Practical Privacy Filters and Odometers with Rényi Differential Privacy and Applications to Differentially Private Deep Learning" [2103.01379]

Source: https://www.emergentmind.com/topics/privacy-odometers