Corr-RR: Correlated Randomized Response

• Corr-RR is a local differential privacy mechanism for multi-attribute frequency estimation that leverages full-budget perturbation and correlation-based reconstruction.
• It employs a two-phase process where a small subset learns interattribute dependencies and the remaining users apply full budget to one attribute for improved utility.
• The mechanism reduces mean squared error compared to standard methods, making it especially effective in high-dimensional and strongly correlated settings.

Correlated Randomized Response (Corr-RR) is a correlation-aware local differential privacy mechanism for multi-attribute frequency estimation. In its explicit recent formulation, it is designed for records with multiple categorical attributes and addresses the utility loss that arises when standard local differential privacy methods either split the privacy budget across all attributes or perturb each attribute independently. Corr-RR uses a small subset of users to learn interattribute dependencies privately and then lets each remaining user perturb a single randomly selected attribute with the full privacy budget, reconstructing the other attributes from the learned dependencies without additional privacy cost (Seeam et al., 23 Jul 2025). Conceptually, it extends the older randomized response lineage in which privacy is understood through local randomization, posterior ambiguity, deniability, and matrix-based misclassification mechanisms (Domingo-Ferrer et al., 2018).

1. Definition and data model

Corr-RR is studied for multi-attribute categorical data. Each user $u_i$ holds a record

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$

where attribute $X_j$ takes values from a finite domain $\mathcal{D}_j$. The aggregation target is the marginal frequency

$$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$

The privacy model is local differential privacy: a mechanism $\mathcal M$ is $\epsilon$-LDP if, for all inputs $x,x'$ and outputs $y$,

$$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$

The stated motivation is that standard multi-attribute LDP methods lose utility in two characteristic ways. Split-budget methods divide $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$0 across $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$1 attributes, so each attribute receives only $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$2, while random-sampling-plus-fake-data methods allocate the full budget to one attribute but often impute the rest in a way that is poorly aligned with the real joint structure. Corr-RR is introduced precisely to exploit natural interattribute correlations under the same nominal privacy budget (Seeam et al., 23 Jul 2025).

The mechanism is therefore not merely a different perturbation kernel. It is a reconstruction strategy built around two principles already visible in the problem statement: full-budget privatization of a small part of each record and correlation-guided inference for the remainder. This makes Corr-RR especially targeted at high-dimensional settings in which independent local randomization causes severe noise accumulation (Seeam et al., 23 Jul 2025).

2. Randomized response foundations and conceptual lineage

Classical randomized response in the cited literature is formulated as a local mechanism for a sensitive attribute $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$3 with $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$4 possible values and reported value $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$5, linked by a transition matrix

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$6

If the true population proportions are $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$7, then the response distribution is

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$8

When $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$9 is nonsingular, the true proportions can be estimated unbiasedly by

$X_j$0

This matrix view is central because it treats randomized response as a general local misclassification channel rather than as a special-purpose survey trick (Domingo-Ferrer et al., 2018).

The same paper reframes randomized response as a deniability mechanism. For a reported value $X_j$1, Bayes’ rule yields

$X_j$2

If the posterior on an embarrassing value remains below certainty, the respondent can deny possessing that value. Deniability is quantified by the conditional Shannon entropy

$X_j$3

with maximum $X_j$4 when the posterior is uniform. The paper further notes a perfect-secrecy special case: if the probabilities in each column of $X_j$5 are identical, then $X_j$6 for any $X_j$7, but $X_j$8 becomes singular, so the unbiased inverse estimator cannot be computed. The same work also presents randomized response as a local version of PRAM and uses deniability and permutation as unifying ideas connecting randomized response, differential privacy, PRAM, and $X_j$9-closeness (Domingo-Ferrer et al., 2018).

This lineage is relevant to Corr-RR because it identifies the structural object that correlated variants modify: the randomization channel. In a related optimization result for binary-output generalized randomized response, the unique optimal mechanism under strict $\mathcal{D}_j$0-DP is the symmetric channel

$\mathcal{D}_j$1

showing that classical RR already admits an explicit differential-privacy-optimal form (Holohan et al., 2016). For discrete-valued sensitive variables, a further generalized mechanism uses

$\mathcal{D}_j$2

which again makes RR a symmetric misclassification model with unbiased inversion formulas for the category proportions and the population mean (Bose, 2013). Corr-RR retains this local-channel perspective but departs from the independent-per-attribute assumption by making inference depend on learned attribute dependencies.

3. Two-phase Corr-RR mechanism

Corr-RR operates in two phases. In Phase I, a small subset of $\mathcal{D}_j$3 users reports all $\mathcal{D}_j$4 attributes, and each attribute is perturbed separately using generalized randomized response with privacy budget $\mathcal{D}_j$5. For attribute $\mathcal{D}_j$6,

$\mathcal{D}_j$7

where $\mathcal{D}_j$8. From these noisy reports the server forms unbiased marginal estimates

$\mathcal{D}_j$9

and uses them to derive a correlation-aware probability $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$0 for each attribute pair. The interpretation given is directional: positive alignment pushes $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$1 toward $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$2, negative relation pushes it toward $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$3, and little directional information places it near $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$4 (Seeam et al., 23 Jul 2025).

In Phase II, the remaining $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$5 users each select one attribute uniformly from $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$6, perturb only that attribute using the full budget $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$7, and generate the unselected attributes from the learned dependency model. The selected attribute uses

$$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$8

Each other attribute $$f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v).$$9 is then indirectly generated from the already perturbed value $\mathcal M$0 via

$\mathcal M$1

The critical design point is that this indirect generation is post-processing of an already privatized value, so it does not consume additional privacy budget (Seeam et al., 23 Jul 2025).

Corr-RR therefore differs from standard per-attribute RR in two precise respects. First, it concentrates the full privacy budget on one attribute for most users instead of fragmenting the budget over all attributes. Second, it replaces fake-data completion with dependency-guided reconstruction learned privately from the data itself (Seeam et al., 23 Jul 2025).

4. Estimation, optimization, and privacy guarantee

The phase-wise estimators in Corr-RR are generalized RR estimators. In Phase II, for a value $\mathcal M$2,

$\mathcal M$3

The final estimator combines the two phases by a weighted average: $\mathcal M$4 This preserves the usual unbiased GRR-style correction structure while allocating the noisy sample budget asymmetrically across users (Seeam et al., 23 Jul 2025).

A substantial part of the mechanism is the choice of $\mathcal M$5. The paper models the Phase-II mean squared error for a categorical value $\mathcal M$6 as

$\mathcal M$7

where

$\mathcal M$8

$\mathcal M$9

$\epsilon$0

$\epsilon$1

Averaging over all values yields a quadratic objective in $\epsilon$2, and the stated optimizer is

$\epsilon$3

The endpoints $\epsilon$4 and $\epsilon$5 are also evaluated, and the best of $\epsilon$6 is chosen (Seeam et al., 23 Jul 2025).

The privacy theorem is explicit: Corr-RR satisfies $\epsilon$7-LDP. The proof uses three standard facts. Phase I is $\epsilon$8-LDP by sequential composition because each user perturbs all $\epsilon$9 attributes with GRR using $x,x'$0 per attribute. Phase II is $x,x'$1-LDP because one attribute is perturbed with budget $x,x'$2 and the rest are produced by post-processing. Since the Phase-I and Phase-II user sets are disjoint, the combined mechanism remains $x,x'$3-LDP by parallel composition. The paper also notes a privacy amplification viewpoint for random attribute selection in Phase II,

$x,x'$4

while emphasizing that the formal guarantee remains $x,x'$5-LDP (Seeam et al., 23 Jul 2025).

5. Pairwise correlated randomized response: Joint RR

A distinct but closely related correlated randomized response construction is Joint Randomized Response (JRR), which addresses binary frequency estimation under local differential privacy. JRR divides users into disjoint groups of two and introduces dependence between the two users’ perturbations while preserving the same one-user marginal behavior as classical RR. If pair $x,x'$6 contains users $x,x'$7 and $x,x'$8, then the joint truthfulness vector $x,x'$9 is sampled according to

$y$0

$y$1

$y$2

$y$3

where $y$4, $y$5, and

$y$6

Each user then reports

$y$7

When $y$8, the mechanism reduces to independent classical RR within each pair (Zheng et al., 15 May 2025).

The defining property is that the marginals remain unchanged: $y$9 Thus each individual report looks exactly like classical RR in isolation, while the pairwise dependence can reduce estimator variance through negative covariance. The unbiased estimator remains

$$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$0

with

$$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$1

The variance is

$$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$2

which collapses to classical RR when $$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$3. The paper explains the utility gain through

$$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$4

with JRR choosing the joint distribution so that $$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$5 is negative in the useful regimes. In the illustrative example with $$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$6, the independent RR variance for estimating $$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$7 is $$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$8, while the correlated example drops it to about $$\Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y).$$9 (Zheng et al., 15 May 2025).

The privacy analysis is correspondingly subtler than in independent RR. JRR analyzes a conditional privacy notion under possible collusion and proves

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$00

with

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$01

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$02

The guarantee relies on hiding group membership. If the collector knew which two users were paired, and especially if one member colluded, the correlation could reveal extra information about the partner. This hidden-pair condition is therefore central rather than incidental (Zheng et al., 15 May 2025).

JRR is not identical to multi-attribute Corr-RR, but it is a direct correlated randomized response scheme in the same broad family: it preserves RR-like marginals while engineering dependence to improve aggregate estimation.

6. Empirical behavior, limitations, and terminological boundaries

Empirically, Corr-RR is evaluated on synthetic data with $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$03 or $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$04 attributes, binary domain $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$05, categorical domain $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$06, correlations $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$07, and $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$08, as well as on the real-world Clave, Nursery, and Mushroom datasets. The metric is

$\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$09

The reported pattern is consistent: Corr-RR usually achieves the lowest MSE, and its gains are strongest when the number of attributes is larger, $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$10 is small, and correlations are stronger. In synthetic experiments with $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$11, Corr-RR can reduce MSE by about $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$12 versus SPL at $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$13 in binary $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$14-attribute settings, and in $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$15-attribute categorical settings it can achieve over $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$16 reduction relative to SPL at $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$17 under strong correlation. On Clave, Nursery, and Mushroom, it outperforms SPL, RS+FD, and RS+RFD, with especially large gains on Mushroom at low $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$18 (Seeam et al., 23 Jul 2025).

The mechanism also has explicit limitations. If attributes are nearly independent, the reconstruction step has less signal to exploit. If $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$19 is too small, the correlation estimate $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$20 may be inaccurate; if $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$21 is too large, too many users are spent in the noisier Phase-I regime. The experiments identify a sweet spot around $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$22. The paper also notes that some correlation patterns are complex and may not be captured perfectly by a single correlation parameter, motivating the sketched alternative “Cond-RR,” and that the exact $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$23 derivation is more straightforward when domains are aligned (Seeam et al., 23 Jul 2025).

A useful way to situate Corr-RR is to contrast it with adjacent mechanisms:

Mechanism	Dependence structure	Primary setting
Standard RR	Independent per-attribute or per-user perturbation	Local randomization and survey/privacy channels
JRR	Pairwise correlated perturbations within hidden user pairs	Binary frequency estimation under LDP
Corr-RR	Interattribute dependency learning and reconstruction from one privatized attribute	Multi-attribute frequency estimation under LDP

Several neighboring topics are often conflated with Corr-RR but are technically distinct. The readout-error mitigation protocol based on correlated POVM reconstruction in quantum devices is related only at a high conceptual level because it corrects correlated distortion in outputs, but it is a non-randomized detector-tomography and inversion method rather than a randomized-response mechanism (Aasen et al., 31 Mar 2025). The improved randomized response technique for two sensitive attributes is a joint survey design for estimating $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$24, $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$25, and $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$26, but its “correlation” is primarily the association between the sensitive attributes themselves rather than explicit correlation in the randomization process (Ewemooje et al., 2016). Private machine learning via randomized response generalizes the idea still further by learning from one noisy release per datapoint through a known corruption channel $\mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}),$27, but it does not define Corr-RR as a named multi-attribute LDP mechanism (Barber, 2020).

The resulting picture is precise. Corr-RR, in the strict contemporary sense, is the two-phase multi-attribute LDP mechanism that privately learns dependencies and then reconstructs unreported attributes from one fully privatized attribute per user. In a broader methodological sense, it belongs to a family of correlated randomized response constructions that preserve local privacy behavior while using carefully designed dependence to recover utility lost by independent perturbation.