Deep Variational Privacy Funnel

• Deep Variational Privacy Funnel is a family of privacy-preserving representation learning methods that use deep stochastic encoders, variational bounds, and adversarial training to balance utility and privacy.
• The approach approximates mutual information and other complex metrics with tractable variational bounds, enabling optimization of privacy leakage versus data utility in high-dimensional models.
• Architectural implementations range from Gaussian latent models to distributed latent filters, demonstrating empirical trade-offs across modalities like images, text, and tabular data.

Searching arXiv for recent and foundational papers on Deep Variational Privacy Funnel and closely related privacy-funnel formulations. Deep Variational Privacy Funnel denotes a family of privacy-preserving representation-learning methods that instantiate the information-theoretic Privacy Funnel with deep stochastic encoders, variational bounds, and, in several formulations, adversarial training. Across these formulations, one observes private information $S$, observed data $X$, utility information $Y$ or $U$, and a released representation $Z$, and seeks an encoder or release mechanism that minimizes leakage such as $I(S;Z)$ while retaining utility such as $I(Y;Z)$, $I(X;Z)$, or $I(X;Y\mid S)$ under tractable learning objectives (Razeghi et al., 2024, Rodríguez-Gálvez et al., 2020). In end-to-end settings this yields latent Gaussian models with utility and privacy decoders; in distributed settings a shared VAE produces a compact latent code and a per-user filter perturbs that code under a divergence budget; and in more recent extensions related bottleneck constructions are applied to federated release and noisy transformer embeddings with Rényi-divergence and Bayesian Differential Privacy guarantees (Chen et al., 2019, Alsulaimawi et al., 4 May 2026, Zein et al., 5 Jan 2026).

1. Information-theoretic formulation

The classical Privacy Funnel considers a stochastic release mechanism from useful data to a released variable and optimizes a privacy–utility trade-off in mutual-information terms. In the formulation summarized by Rodríguez-Gálvez et al., the classical problem is

$$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$

where $X$0 is the private attribute, $X$1 is the public data, and $X$2 is the released representation (Rodríguez-Gálvez et al., 2020). Their Conditional Privacy Funnel refines this by preserving only the non-private part of $X$3: $X$4

The end-to-end DVPF formulation in face-recognition and representation-learning settings is typically written as

$X$5

or, equivalently,

$X$6

with a Lagrange multiplier controlling the privacy–utility trade-off (Razeghi et al., 2024). Razeghi et al. further separate a discriminative formulation,

$X$7

from a generative formulation,

$X$8

where $X$9 is a synthetic output (Razeghi et al., 2024).

An alternative but equivalent operational view appears in the Gaussian Privacy Protector, where the constrained problem

$Y$0

is rewritten as

$Y$1

with $Y$2 trading off privacy against utility (Alsulaimawi et al., 4 May 2026). Earlier adversarial neural implementations use the same privacy-funnel logic with a distortion penalty: $Y$3 for a randomized mechanism $Y$4 (Tripathy et al., 2017).

These formulations share the same structural principle: privacy is defined as low information flow from sensitive attributes to the released representation, while utility is defined as retention of task-relevant or reconstruction-relevant information.

2. Variational bounds and tractable objectives

Direct evaluation of $Y$5, $Y$6, $Y$7, or $Y$8 is generally intractable in high-dimensional models, so DVPF replaces these quantities with variational upper or lower bounds. In the supervised DVPF summarized in the face-recognition paper, the utility term admits the lower bound

$Y$9

which leads to maximizing

$U$0

The same work gives two complementary bounds on the leakage term $U$1: a classification-style bound using a private-attribute decoder $U$2, and a complexity–uncertainty bound based on the identity

$U$3

with a KL-based information-complexity term and a conditional-decoder uncertainty term (Razeghi et al., 2024).

The Conditional Privacy Funnel of Rodríguez-Gálvez et al. uses an encoder $U$4, a marginal proxy $U$5, and a conditional decoder $U$6 to bound

$U$7

and

$U$8

Substituting these into the Lagrangian yields the stochastic loss

$U$9

with $Z$0 (Rodríguez-Gálvez et al., 2020).

The Gaussian Privacy Protector introduces variational families $Z$1 and $Z$2 and derives a lower bound on $Z$3 and an upper bound on $Z$4, producing the saddle-point objective

$Z$5

Here the privacy term is adversarial and the utility term is predictive, but both remain grounded in mutual-information bounds (Alsulaimawi et al., 4 May 2026).

Privacy-Preserving Adversarial Networks use the same variational device in an earlier form. Introducing an adversary $Z$6 yields

$Z$7

while an optional decoder $Z$8 lower-bounds $Z$9 (Tripathy et al., 2017). In this sense, DVPF can be viewed as a deep variational generalization of the Privacy Funnel in which privacy leakage is approximated either through adversarial classification, decoder-based bounds, or both.

3. Architectural realizations

A common end-to-end DVPF architecture uses a Gaussian encoder $I(S;Z)$0, utility decoder $I(S;Z)$1, private-attribute decoder $I(S;Z)$2, and a prior or proposal $I(S;Z)$3 that is either fixed isotropic $I(S;Z)$4 or learned adversarially (Razeghi et al., 2024). In the generative variant, a decoder or generator reconstructs or synthesizes $I(S;Z)$5 from $I(S;Z)$6, and privacy is enforced either on the latent code or on the generated output (Razeghi et al., 2024).

A distinct architectural pattern, introduced for distributed user customization, decouples representation learning from privatization. A VAE first learns a shared latent representation through

$I(S;Z)$7

and once trained the encoder $I(S;Z)$8 is frozen and shared by all users (Chen et al., 2019). Each user then trains only a small generative filter $I(S;Z)$9 in latent space. In the detailed formulation of Chen, Navidi and Rajagopal, the filter uses the deterministic posterior mean embedding $I(Y;Z)$0, auxiliary noise $I(Y;Z)$1, and a one-hot private label $I(Y;Z)$2, and produces

$I(Y;Z)$3

The filter is trained against a privacy adversary and a utility predictor under an $I(Y;Z)$4-divergence budget between $I(Y;Z)$5 and $I(Y;Z)$6 (Chen et al., 2020).

The same decoupled design appears in a slightly more general notation as a perturbation filter $I(Y;Z)$7 mapping $I(Y;Z)$8, with a typical architecture

$I(Y;Z)$9

and output $I(X;Z)$0 (Chen et al., 2019).

In federated settings, the Gaussian Privacy Protector replaces the shared fixed VAE with client-side stochastic encoders. Each client retains raw $I(X;Z)$1 and sensitive labels $I(X;Z)$2 locally, sends only sanitized $I(X;Z)$3 to the server, and updates its own adversary $I(X;Z)$4 without sharing sensitive labels (Alsulaimawi et al., 4 May 2026).

For text, the Nonparametric Variational Information Bottleneck inserts an “NVIB layer” into a frozen BERT encoder. The posterior is a Dirichlet-process mixture of impulse vectors,

$I(X;Z)$5

with parameters $I(X;Z)$6 produced by a small projection network. A sampled noisy representation $I(X;Z)$7 is fed into a denoising multi-head attention block, and the residual skip around that block is removed to ensure that all information passes through the noisy bottleneck (Zein et al., 5 Jan 2026).

Taken together, these implementations show that DVPF is not tied to a single architecture: it appears as end-to-end latent Gaussian models, decoder-conditioned CPF models, per-user latent filters, federated encoders, and nonparametric transformer bottlenecks.

4. Optimization, estimation, and privacy accounting

Optimization is typically cast as alternating minimization or minimax training. In PPAN one solves

$I(X;Z)$8

alternating adversary updates with mechanism updates (Tripathy et al., 2017). The user-customized latent filter uses a robust min–max objective

$I(X;Z)$9

or its Lagrangian-relaxed version with weight $I(X;Y\mid S)$0 on the divergence term (Chen et al., 2019). GPP uses two phases: first training the adversary and utility decoder on encoded samples, then updating the encoder using the combined loss with CE terms and Gaussian-prior KL regularization (Alsulaimawi et al., 4 May 2026).

The reparameterization trick is standard in Gaussian-latent variants: $I(X;Y\mid S)$1 allowing gradients to propagate through stochastic samples (Alsulaimawi et al., 4 May 2026). The Conditional Privacy Funnel likewise uses $I(X;Y\mid S)$2 with $I(X;Y\mid S)$3 (Rodríguez-Gálvez et al., 2020).

Empirical leakage is commonly assessed through mutual-information estimators. The distributed latent-filter framework uses a $I(X;Y\mid S)$4-nearest-neighbor estimator from Gao et al. (2015),

$I(X;Y\mid S)$5

and also validates with sample-based variational bounds of the form

$I(X;Y\mid S)$6

(Chen et al., 2019). Chen, Navidi and Rajagopal report the same estimator in a Kozachenko–Leonenko style form for $I(X;Y\mid S)$7 (Chen et al., 2020).

A central distinction in this literature is between mutual-information privacy objectives and formal differential privacy guarantees. In the linear-filter case, Chen, Navidi and Rajagopal show that if

$I(X;Y\mid S)$8

with $I(X;Y\mid S)$9 and $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$0, then under Theorem 3 the filter yields $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$1-Rényi differential privacy in $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$2 under an explicit bound involving $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$3, $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$4, and $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$5 (Chen et al., 2020). The transformer-based NVIB method goes further by directly controlling Rényi divergence

$$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$6

and converting it to a Bayesian Differential Privacy guarantee using Theorem 2 of Triastcyn and Faltings (ICML 2020) (Zein et al., 5 Jan 2026).

Federated privacy accounting is treated differently in GPP. Under IID data and an honest aggregator, the bound

$$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$7

shows how client-level privacy composes with any residual leakage carried by utility labels (Alsulaimawi et al., 4 May 2026).

A common misconception is that any DVPF objective automatically yields differential privacy. The cited formulations do not support that conclusion in general. Formal Rényi-DP or BDP guarantees arise only in models that explicitly analyze divergence, such as the linear-filter case or the NVIB transformer mechanism (Chen et al., 2020, Zein et al., 5 Jan 2026).

5. Empirical behavior across modalities

Empirical studies consistently report a trade-off curve rather than privacy improvement at fixed utility for all operating points. On MNIST, in the distributed latent-filter setup with digit identity privatized and “circle vs non-circle” preserved, the raw encoder output $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$8 yields private accuracy $$\min_{P_{Y|X}} I(S;Y) \quad\text{s.t.}\quad I(X;Y)\ge r',$$9 and utility $X$00, while after the filter with $X$01 the private-label accuracy drops to $X$02 and utility stays $X$03; UMAP shows ten distinct clusters collapsing into two well-separated “circles vs non-circles” (Chen et al., 2019). In a second MNIST setting that privatizes “$X$04” and preserves parity, private-label accuracy moves from $X$05 as $X$06 grows, while utility stays around $X$07 (Chen et al., 2019). Chen, Navidi and Rajagopal report a closely related MNIST experiment in which as the KL-budget $X$08 grows $X$09, private-ID accuracy drops from $X$10 while utility stays above $X$11, with a knee around $X$12 (Chen et al., 2020).

On tabular data, the UCI-Adult benchmark privatizes gender and preserves income. The reported comparison is:

Model	Privacy metric	Utility metric
Plain VAE	private acc. $X$13, private AUROC $X$14	utility acc. $X$15, utility AUROC $X$16
VFAE	private acc. $X$17, private AUROC $X$18	utility acc. $X$19, utility AUROC $X$20
LMIFR	private acc. $X$21, private AUROC $X$22	utility acc. $X$23, utility AUROC $X$24
filter	private acc. $X$25, private AUROC $X$26	utility acc. $X$27, utility AUROC $X$28

The reported interpretation is that the filter achieves the lowest gender leakage with only minor loss in income prediction (Chen et al., 2019). In UCI-Abalone, with utility Rings $X$29 and private Sex, a KL-budget $X$30 lowers private-sex accuracy from $X$31 to $X$32 while utility stays $X$33 (Chen et al., 2020).

On CelebA, multiple papers report similar patterns. In the distributed latent-filter setting, classifiers achieve $X$34 accuracy on raw pixels and $X$35 on raw 50-D VAE embeddings for eight private attributes; after filtering, private-attribute accuracy falls to $X$36 on average, close to random-guess $X$37, while smiling accuracy drops only $X$38 from $X$39 (Chen et al., 2019). Chen, Navidi and Rajagopal report that on a 100-D VAE code, private-label accuracy is $X$40 before filtering and $X$41 after filtering on average, while smiling remains $X$42 (Chen et al., 2020). The Gaussian Privacy Protector gives, for Smiling as utility and Gender as sensitive attribute with $X$43 and $X$44, utility $X$45 AUC and adversary $X$46 AUC under a 48$X$47 compression from $X$48 dimensions (Alsulaimawi et al., 4 May 2026).

On MNIST, GPP reports utility within roughly one percentage point of an unconstrained autoencoder baseline while reducing the adversary’s AUC to near random guessing: No-Privacy AE gives Utility $X$49 AUC and Adversary $X$50 AUC, whereas GPP at $X$51 gives Utility $X$52 and Adversary $X$53 (Alsulaimawi et al., 4 May 2026). On HAPT-Recognition, centralized GPP reaches Utility $X$54, Adversary $X$55, and the distributed version with five clients reaches $X$56, or $X$57 with heterogeneous $X$58 (Alsulaimawi et al., 4 May 2026).

Face-recognition experiments emphasize the cost of aggressive privacy protection. Before DVPF, raw embeddings carry nearly full sensitive information, with $X$59 bits and accuracy $X$60, and $X$61 bits with accuracy $X$62 (Razeghi et al., 2024). After DVPF on IResNet-50 embeddings, one reported setting gives at $X$63 a drop to $X$64 bits, $X$65, and TMR $X$66 versus $X$67 baseline; at $X$68, $X$69 bits, $X$70, and TMR $X$71 (Razeghi et al., 2024). A related face-recognition study reports that as $X$72 grows from $X$73, attribute-classification accuracy drops from $X$74 and utility degrades from $X$75 in DisPF or $X$76 in GenPF, with stronger effects at lower latent dimension $X$77 (Razeghi et al., 2024).

In text classification, NVIB-based noisy transformer embeddings show a privacy–utility frontier on GLUE. For MRPC, the best reported NVDP result is $X$78 accuracy with $X$79 and BDP $X$80, compared with VTDP at $X$81 and VIB-fixed at $X$82; for SST-2, NVDP gives $X$83 (Zein et al., 5 Jan 2026). The reported accuracy-versus-Bayesian-DP curves show that for any target $X$84, NVDP consistently achieves $X$85–$X$86 points higher accuracy than VTDP and $X$87–$X$88 lower Rényi divergence than single-vector VIB baselines (Zein et al., 5 Jan 2026).

6. Relations to adjacent frameworks and recurrent points of confusion

DVPF sits at the intersection of Privacy Funnel, Variational Information Bottleneck, VAE-style latent-variable modeling, adversarial learning, and, in recent extensions, differential privacy analysis. Rodríguez-Gálvez et al. state that the approach can be comfortably incorporated into common representation learning algorithms such as the VAE, the $X$89-VAE, the VIB, or the nonlinear IB (Rodríguez-Gálvez et al., 2020). Razeghi et al. explicitly connect DVPF to VAEs, GANs, and Diffusion models, noting that the first two terms of their discriminative and generative objectives coincide with a standard VAE ELBO and that latent-space and output-space matching can be implemented adversarially (Razeghi et al., 2024).

One recurrent design issue concerns where to inject the sensitive attribute $X$90. The Conditional Privacy Funnel emphasizes that, unlike earlier VAE-based PF variants, DVPF does not feed $X$91 into the encoder $X$92 but only into the decoder. The stated reason is that this guarantees the encoder must remove all $X$93-information from $X$94 (Rodríguez-Gálvez et al., 2020). In contrast, the user-customized latent-filter models do condition the small filter on the private label because the goal there is post hoc, user-specific perturbation of an already learned shared latent space rather than end-to-end encoder learning (Chen et al., 2019, Chen et al., 2020).

Another recurrent misconception is that local or federated storage alone solves the privacy problem. The federated GPP paper states that in privacy-sensitive deployments such as medical sensors, IoT devices, and wearables, the protection offered by keeping data local is incomplete because gradients, model updates, and the released representations themselves can leak sensitive attributes (Alsulaimawi et al., 4 May 2026). This motivates DVPF-style sanitization even when raw data never leave the device.

The literature also distinguishes discriminative protection from generative protection. DisPF obfuscates the latent code $X$95, whereas GenPF synthesizes $X$96 with privacy constraints on $X$97 (Razeghi et al., 2024). This suggests that “privacy-preserving representation learning” and “privacy-preserving data generation” are not competing notions within this family but two realizations of the same information-theoretic trade-off.

A final point concerns scope. The empirical record summarized across Adult, Colored-MNIST, COMPAS, MNIST, CelebA, HAPT, face-recognition benchmarks, and GLUE shows that DVPF-like methods can support classification, reconstruction, generation, federated release, and sanitized transformer embeddings (Rodríguez-Gálvez et al., 2020, Razeghi et al., 2024, Alsulaimawi et al., 4 May 2026, Zein et al., 5 Jan 2026). A plausible implication is that the principal modeling choice is less the data modality than the definition of utility, the threat model, and the form of privacy accounting—mutual information, adversarial predictability, divergence budgets, or explicit Rényi/Bayesian DP guarantees.