Decentralized Differentially Private Power Method

• The paper presents decentralized iterative methods that fuse power iteration with calibrated Gaussian noise to guarantee (ε,δ)-differential privacy without a central curator.
• It distinguishes two main variants: one for PCA using row-wise data partitions with gossip consensus and one for power-controlled decentralized optimization over multicast networks.
• Experimental results demonstrate significant utility gains over local-DP methods, with rapid convergence and reduced approximation error in moderate privacy regimes.

Decentralized Differentially Private Power Method (D-DP-PM) denotes a class of decentralized iterative algorithms that combine power-style updates, inter-agent mixing, and calibrated Gaussian perturbations to satisfy $(\epsilon,\delta)$-differential privacy without requiring a central curator. In the 2025 arXiv literature, the acronym is used in two distinct senses: a fully decentralized power method for principal component analysis under row-wise data partitioning, and a power-controlled decentralized learning rule over heterogeneous multicast networks; a closely related antecedent is the decentralized differentially-private randomized power method for private eigenspace approximation (Campbell et al., 30 Jul 2025, Ziaeddini et al., 25 Sep 2025, Nicolas et al., 2024).

1. Scope and nomenclature

The name D-DP-PM is not attached to a single canonical algorithm. Rather, it has been used for closely related but substantively different decentralized privacy-preserving iterative schemes.

Variant	Core task	Distinguishing features
D-DP-PM for PCA (Campbell et al., 30 Jul 2025)	Estimate the top-$r$ eigenvectors of $\Sigma=X^\top X$	Row-wise partitioning, undirected connected graph, doubly stochastic mixing, gossip consensus, noisy power iteration
Power-controlled D-DP-PM (Ziaeddini et al., 25 Sep 2025)	Minimize $F(x)=\sum_{i=1}^K f_i(x)$ over a multicast network	Directed strongly connected graph, row-stochastic mixing induced by channel gains, joint transmit-power/noise control, regret analysis
Decentralized DP randomized power method (Nicolas et al., 2024)	Approximate the top-$k$ eigenspace of $A=\sum_{i=1}^s A^{(i)}$	Secure aggregation or GOPA, improved sensitivity analysis, centralized-equivalent privacy and utility

This terminological overlap matters because the mathematical object being iterated differs across the variants. In the PCA-oriented formulation, the power iteration targets the dominant eigenspace of a global covariance matrix. In the heterogeneous multicast formulation, the update is a differentially private decentralized stochastic-gradient step in which power allocation and communication topology jointly determine the effective mixing matrix. A plausible implication is that D-DP-PM should be read as a descriptor of method family rather than as the name of a unique protocol.

2. PCA-oriented D-DP-PM: model and update dynamics

In the PCA formulation, the global data matrix is $X\in\mathbb{R}^{n\times d}$, with rows partitioned sample-wise among $m$ agents, where agent $i$ holds $X_i\in\mathbb{R}^{n_i\times d}$ and $r$0 (Campbell et al., 30 Jul 2025). The objective is to recover the top-$r$1 eigenvectors of the global covariance

$r$2

with eigendecomposition $r$3, $r$4, and $r$5. The principal eigenvector is denoted $r$6.

The communication substrate is an undirected, connected graph $r$7 with mixing matrix $r$8 satisfying four structural conditions: $r$9 iff $\Sigma=X^\top X$0, $\Sigma=X^\top X$1, $\Sigma=X^\top X$2, $\Sigma=X^\top X$3, and $\Sigma=X^\top X$4. The paper also introduces the consensus-modified covariance

$\Sigma=X^\top X$5

where $\Sigma=X^\top X$6 and $\Sigma=X^\top X$7 is the number of gossip steps. Assumption 4 requires

$\Sigma=X^\top X$8

so that the spectral gap of $\Sigma=X^\top X$9 remains positive.

The decentralized iteration computes one eigenvector at a time. Each agent maintains a local vector $F(x)=\sum_{i=1}^K f_i(x)$0. Initialization uses independent Gaussian draws,

$F(x)=\sum_{i=1}^K f_i(x)$1

At iteration $F(x)=\sum_{i=1}^K f_i(x)$2, agent $F(x)=\sum_{i=1}^K f_i(x)$3 first performs a local projection,

$F(x)=\sum_{i=1}^K f_i(x)$4

then runs $F(x)=\sum_{i=1}^K f_i(x)$5 gossip steps to obtain

$F(x)=\sum_{i=1}^K f_i(x)$6

and finally updates via the noisy power rule

$F(x)=\sum_{i=1}^K f_i(x)$7

After $F(x)=\sum_{i=1}^K f_i(x)$8 iterations, the agents share $F(x)=\sum_{i=1}^K f_i(x)$9, stack the blocks into

$k$0

and normalize:

$k$1

Subsequent components are extracted by deflation, with each agent updating

$k$2

Consensus quality is controlled by the spectral contraction of $k$3. The paper states that

$k$4

so increasing the gossip depth $k$5 suppresses the network-induced approximation error.

3. Privacy mechanism in the decentralized PCA method

The privacy model assumes that adjacent datasets differ in one row and that the $k$6 sensitivity is $k$7 after normalization (Campbell et al., 30 Jul 2025). No node colludes with an external party, but nodes are honest-but-curious about others’ data. The sources of randomness are the initial vectors $k$8 and the per-iteration Gaussian perturbations $k$9.

Each release $A=\sum_{i=1}^s A^{(i)}$0 has $A=\sum_{i=1}^s A^{(i)}$1-sensitivity $A=\sum_{i=1}^s A^{(i)}$2. Under the Gaussian mechanism, a single release is $A=\sum_{i=1}^s A^{(i)}$3-DP if

$A=\sum_{i=1}^s A^{(i)}$4

Because each agent releases $A=\sum_{i=1}^s A^{(i)}$5 vectors, privacy accounting is performed through advanced composition via Rényi-DP. The stacked release

$A=\sum_{i=1}^s A^{(i)}$6

is jointly Gaussian with mean $A=\sum_{i=1}^s A^{(i)}$7 and covariance $A=\sum_{i=1}^s A^{(i)}$8, and for an adjacent dataset $A=\sum_{i=1}^s A^{(i)}$9, the corresponding release $X\in\mathbb{R}^{n\times d}$0 satisfies

$X\in\mathbb{R}^{n\times d}$1

The paper therefore chooses the noise schedule so that each agent’s $X\in\mathbb{R}^{n\times d}$2 and the composed $X\in\mathbb{R}^{n\times d}$3 values sum to $X\in\mathbb{R}^{n\times d}$4.

A practical allocation used in the paper is uniform budgeting across the $X\in\mathbb{R}^{n\times d}$5 releases:

$X\in\mathbb{R}^{n\times d}$6

This construction is explicitly end-to-end: privacy is attached to the full vector of releases rather than to an isolated update. A common misconception is that random initialization alone supplies privacy. The formulation indicates otherwise: the initial Gaussian draw contributes randomness, but the prescribed $X\in\mathbb{R}^{n\times d}$7 guarantee is enforced through calibrated per-iteration Gaussian noise and Rényi-based composition.

4. Convergence, topology dependence, and empirical behavior of the PCA method

The convergence analysis treats the full network stack $X\in\mathbb{R}^{n\times d}$8 as multivariate Gaussian,

$X\in\mathbb{R}^{n\times d}$9

where $m$0 depends explicitly on $m$1, $m$2, $m$3, and the spectrum of $m$4 (Campbell et al., 30 Jul 2025). Utility is measured through the principal-angle error

$m$5

Let $m$6 be the principal eigenvector of $m$7 with eigenvalue $m$8, and let $m$9 be the second largest eigenvalue. Theorem 2 introduces

$i$0

where $i$1 is chosen to satisfy the Hanson–Wright tail condition, and

$i$2

With probability at least $i$3,

$i$4

The bound separates three effects: concentration of the Gaussian iterate, consensus error through $i$5, and residual spectral error through $i$6. Corollary 1 states that if $i$7 and $i$8, then

$i$9

so the noise-induced term decays essentially at the usual geometric rate $X_i\in\mathbb{R}^{n_i\times d}$0 up to a constant.

The experimental study uses four row-normalized datasets: Diabetes $X_i\in\mathbb{R}^{n_i\times d}$1, Breast Cancer $X_i\in\mathbb{R}^{n_i\times d}$2, Wine $X_i\in\mathbb{R}^{n_i\times d}$3, and AMI $X_i\in\mathbb{R}^{n_i\times d}$4. The network has $X_i\in\mathbb{R}^{n_i\times d}$5 agents arranged on a ring with

$X_i\in\mathbb{R}^{n_i\times d}$6

Privacy schedules use $X_i\in\mathbb{R}^{n_i\times d}$7 with dataset-specific $X_i\in\mathbb{R}^{n_i\times d}$8 upper bounds. The baseline is Local DP PCA, where each agent perturbs its entire block $X_i\in\mathbb{R}^{n_i\times d}$9 by Gaussian noise $r$00 and then performs a local SVD. In the moderate privacy regime $r$01, D-DP-PM reduces approximation error by up to $r$02 compared with LDP, with stronger advantage in higher-dimensional settings such as Breast Cancer and AMI. Figure 1 further indicates rapid convergence, with most of the loss recovered by approximately $r$03 iterations, and a corresponding increase in $r$04 as iterations are added.

5. Power-controlled D-DP-PM in heterogeneous multicast networks

A second use of D-DP-PM appears in a different problem class: decentralized optimization over a directed, strongly connected multicast graph $r$05 on $r$06 clients (Ziaeddini et al., 25 Sep 2025). If $r$07, client $r$08 can multicast to client $r$09. The associated adjacency matrix $r$10 is row-stochastic:

$r$11

Heterogeneous channel gains enter through

$r$12

where $r$13 is the physical channel gain, $r$14 is the maximum power, and $r$15 is the data-power fraction.

Each client $r$16 has a local $r$17-strongly-convex cost $r$18 over a convex compact $r$19, and the global objective is

$r$20

Differential privacy uses the standard $r$21 definition, and for a vector-valued query $r$22 with sensitivity $r$23, the Gaussian mechanism requires

$r$24

The distinctive feature of this variant is joint control of signal power and injected noise under an energy budget. Each client $r$25 has per-epoch power budget $r$26 and splits it as

$r$27

The transmitted vector is

$r$28

with $r$29. Under the noiseless multiple-access assumption, neighbor $r$30 receives

$r$31

The induced row-stochastic weights are

$r$32

where

$r$33

Each client also maintains an auxiliary state $r$34, initialized by $r$35, to estimate the left Perron vector $r$36 of $r$37. With local gradient $r$38 and learning rate $r$39, the update is

$r$40

The DP leakage from client $r$41 to client $r$42 in epoch $r$43 is

$r$44

In practice, the paper fixes an upper bound $r$45 and solves the linear program

$r$46

Privacy and energy are coupled. By construction, each link $r$47 is $r$48-DP in epoch $r$49, and with $r$50, basic composition yields $r$51-DP over $r$52 epochs. Each client uses exactly $r$53 power per epoch. Larger $r$54 improves signal mixing and reduces optimization error but leaves less power for artificial noise, whereas smaller $r$55 improves privacy at the cost of slower convergence. Under $r$56-strong convexity, bounded gradients $r$57, diameter of $r$58, and the choice $r$59, the expected regret

$r$60

satisfies

$r$61

with $r$62 and $r$63, yielding $r$64 regret.

The numerical study uses $r$65 fully connected clients, two fixed heterogeneous channel-gain matrices drawn Uniform$r$66, and MNIST classification with per-client non-IID splits. Each client trains multinomial logistic regression on an $r$67-train, $r$68-test partition, with $r$69, $r$70, equal maximum power $r$71, and privacy budgets $r$72. The baseline is a modified PED$r$73FL (“mPED$r$74FL”), which enforces a doubly stochastic $r$75 by local compensation and uses separate $r$76 per link. The reported findings are that Alg. 1 converges more quickly in test accuracy versus epoch and achieves higher final accuracy for both $r$77 values; because it uses multicast, it requires only one channel-use per epoch, whereas mPED$r$78FL needs one per neighbor.

6. Relation to the decentralized randomized power method and recurring interpretive issues

The 2024/2025 decentralized differentially-private randomized power method provides an immediate precursor and a useful point of comparison (Nicolas et al., 2024). There, the objective is to approximate the top-$r$79 eigenspace of a real symmetric positive semidefinite matrix $r$80, with user-level partition

$r$81

The algorithm uses an oversampling parameter $r$82, sets $r$83, initializes with $r$84, and iteratively computes

$r$85

followed by QR orthonormalization.

Its main technical refinement is a tighter sensitivity calibration:

$r$86

instead of prior calibrations scaling approximately as $r$87. With

$r$88

the centralized $r$89-step procedure satisfies $r$90-DP. In the decentralized form, users compute

$r$91

and aggregate either through Secure Aggregation or a peer-to-peer gossip averaging protocol (GOPA). The corresponding decentralized noise scale is

$r$92

The paper states that the decentralized protocol has identical privacy cost and utility to the centralized version.

The convergence analysis gives two representative bounds. Under a spectral gap $r$93 and $r$94, a runtime-dependent estimate is

$r$95

A runtime-independent bound replaces the row-norm factor by a coherence quantity:

$r$96

The paper further states that the improved sensitivity can save up to an $r$97 factor in noise variance relative to earlier DP power-iteration analyses.

Two recurring interpretive issues are clarified by these papers. First, decentralized differential privacy is not synonymous with naive local DP. The PCA-oriented D-DP-PM is compared against a local-DP PCA baseline that perturbs each local block before local SVD, and the randomized-power-method paper contrasts its approach with local-DP-only schemes that incur much larger noise; both report superior privacy-utility tradeoffs for decentralized aggregation-based methods (Campbell et al., 30 Jul 2025, Nicolas et al., 2024). Second, the acronym D-DP-PM should not be assumed to specify a unique update equation. In current usage it refers to at least two algorithmic lines: decentralized private eigenspace estimation and power-controlled private decentralized learning. The common structure is the coupling of iterative mixing with Gaussian perturbation under formal $r$98 accounting, but the state variables, network assumptions, and utility criteria differ substantially.