--- title: 'Corr-RR: Correlated Randomized Response' url: https://www.emergentmind.com/topics/correlated-randomized-response-corr-rr type: topic --- # Corr-RR: Correlated Randomized Response Correlated Randomized Response (Corr-RR) is a correlation-aware local differential privacy mechanism for multi-attribute frequency estimation. In its explicit recent formulation, it is designed for records with multiple categorical attributes and addresses the utility loss that arises when standard local differential privacy methods either split the privacy budget across all attributes or perturb each attribute independently. Corr-RR uses a small subset of users to learn interattribute dependencies privately and then lets each remaining user perturb a single randomly selected attribute with the full privacy budget, reconstructing the other attributes from the learned dependencies without additional privacy cost [2507.17516]. Conceptually, it extends the older randomized response lineage in which privacy is understood through local randomization, posterior ambiguity, deniability, and matrix-based misclassification mechanisms [1803.02139]. ## 1. Definition and data model Corr-RR is studied for multi-attribute categorical data. Each user \(u_i\) holds a record \[ \mathbf{x}_i = (x_{i,1}, x_{i,2}, \dots, x_{i,d}), \] where attribute \(X_j\) takes values from a finite domain \(\mathcal{D}_j\). The aggregation target is the marginal frequency \[ f_j(v) = \frac{1}{n}\sum_{i=1}^{n}\mathbb{I}(x_{i,j}=v). \] The privacy model is local differential privacy: a mechanism \(\mathcal M\) is \(\epsilon\)-LDP if, for all inputs \(x,x'\) and outputs \(y\), \[ \Pr[\mathcal{M}(x)=y] \le e^\epsilon \Pr[\mathcal{M}(x')=y). \] The stated motivation is that standard multi-attribute LDP methods lose utility in two characteristic ways. Split-budget methods divide \(\epsilon\) across \(d\) attributes, so each attribute receives only \(\epsilon/d\), while random-sampling-plus-fake-data methods allocate the full budget to one attribute but often impute the rest in a way that is poorly aligned with the real joint structure. Corr-RR is introduced precisely to exploit natural interattribute correlations under the same nominal privacy budget [2507.17516]. The mechanism is therefore not merely a different perturbation kernel. It is a reconstruction strategy built around two principles already visible in the problem statement: full-budget privatization of a small part of each record and correlation-guided inference for the remainder. This makes Corr-RR especially targeted at high-dimensional settings in which independent local randomization causes severe noise accumulation [2507.17516]. ## 2. Randomized response foundations and conceptual lineage Classical randomized response in the cited literature is formulated as a local mechanism for a sensitive attribute \(X\) with \(r\) possible values and reported value \(Y\), linked by a transition matrix \[ P=\big(p_{uv}\big), \qquad p_{uv}=\Pr(Y=v\mid X=u), \quad u,v\in\{1,\dots,r\}. \] If the true population proportions are \(T=(T_1,\dots,T_r)^T\), then the response distribution is \[ \lambda = PT. \] When \(P\) is nonsingular, the true proportions can be estimated unbiasedly by \[ \hat t=(P^T)^{-1}\hat \lambda. \] This matrix view is central because it treats randomized response as a general local misclassification channel rather than as a special-purpose survey trick [1803.02139]. The same paper reframes randomized response as a deniability mechanism. For a reported value \(Y=v\), Bayes’ rule yields \[ \Pr(X=u\mid Y=v)= \frac{\Pr(Y=v\mid X=u)\Pr(X=u)} {\sum_{u'=1}^r \Pr(Y=v\mid X=u')\Pr(X=u')}. \] If the posterior on an embarrassing value remains below certainty, the respondent can deny possessing that value. Deniability is quantified by the conditional Shannon entropy \[ H(X\mid Y=v)=-\sum_{u=1}^r p_{vu}\log_2 p_{vu}, \] with maximum \(\log_2 r\) when the posterior is uniform. The paper further notes a perfect-secrecy special case: if the probabilities in each column of \(P\) are identical, then \(H(X\mid Y=v)=H(X)\) for any \(v\), but \(P\) becomes singular, so the unbiased inverse estimator cannot be computed. The same work also presents randomized response as a local version of PRAM and uses deniability and permutation as unifying ideas connecting randomized response, differential privacy, PRAM, and \(t\)-closeness [1803.02139]. This lineage is relevant to Corr-RR because it identifies the structural object that correlated variants modify: the randomization channel. In a related optimization result for binary-output generalized randomized response, the unique optimal mechanism under strict \(\epsilon\)-DP is the symmetric channel \[ P_\epsilon= \begin{pmatrix} \frac{e^\epsilon}{e^\epsilon+1} & \frac{1}{e^\epsilon+1}\\[4pt] \frac{1}{e^\epsilon+1} & \frac{e^\epsilon}{e^\epsilon+1} \end{pmatrix}, \] showing that classical RR already admits an explicit differential-privacy-optimal form [1612.05568]. For discrete-valued sensitive variables, a further generalized mechanism uses \[ \Pr(R=x_i\mid X=x_j)=p\,\delta_{ij}+\frac{1-p}{m}, \] which again makes RR a symmetric misclassification model with unbiased inversion formulas for the category proportions and the population mean [1303.5172]. Corr-RR retains this local-channel perspective but departs from the independent-per-attribute assumption by making inference depend on learned attribute dependencies. ## 3. Two-phase Corr-RR mechanism Corr-RR operates in two phases. In Phase I, a small subset of \(n_1 \ll n\) users reports all \(d\) attributes, and each attribute is perturbed separately using generalized randomized response with privacy budget \(\epsilon/d\). For attribute \(j\), \[ \Pr(Y_{i,j}=v' \mid x_{i,j}=v)= \begin{cases} p_1=\dfrac{e^{\epsilon/d}}{e^{\epsilon/d}+k-1}, & v'=v,\\[4pt] q_1=\dfrac{1}{e^{\epsilon/d}+k-1}, & v'\neq v, \end{cases} \] where \(k = |\mathcal{D}|\). From these noisy reports the server forms unbiased marginal estimates \[ \hat f_j^{I}(v) = \frac{I_j^{I}(v) - n_1\, q_1}{n_1\bigl(p_1 - q_1\bigr)}, \qquad I_j^{I}(v)=\sum_{i=1}^{n_1}\mathbb{I}(y_{i,j}=v), \] and uses them to derive a correlation-aware probability \(p_y\) for each attribute pair. The interpretation given is directional: positive alignment pushes \(p_y\) toward \(1\), negative relation pushes it toward \(0\), and little directional information places it near \(0.5\) [2507.17516]. In Phase II, the remaining \(n_2=n-n_1\) users each select one attribute uniformly from \([d]=\{1,\dots,d\}\), perturb only that attribute using the full budget \(\epsilon\), and generate the unselected attributes from the learned dependency model. The selected attribute uses \[ \Pr(Y_{i,j}=v' \mid x_{i,j}=v)= \begin{cases} p_2=\dfrac{e^{\epsilon}}{e^{\epsilon}+k-1}, & v'=v,\\[4pt] q_2=\dfrac{1}{e^{\epsilon}+k-1}, & v'\neq v. \end{cases} \] Each other attribute \(X_k\) is then indirectly generated from the already perturbed value \(Y_{i,j}\) via \[ \Pr\!\bigl(Y_{i,k}=v' \mid Y_{i,j}=v\bigr)= \begin{cases} p_y, & v'=v,\\[4pt] 1-p_y, & v'\neq v. \end{cases} \] The critical design point is that this indirect generation is post-processing of an already privatized value, so it does not consume additional privacy budget [2507.17516]. Corr-RR therefore differs from standard per-attribute RR in two precise respects. First, it concentrates the full privacy budget on one attribute for most users instead of fragmenting the budget over all attributes. Second, it replaces fake-data completion with dependency-guided reconstruction learned privately from the data itself [2507.17516]. ## 4. Estimation, optimization, and privacy guarantee The phase-wise estimators in Corr-RR are generalized RR estimators. In Phase II, for a value \(v\), \[ \hat f_j^{\mathrm{II}}(v) = \frac{I_j^{\mathrm{II}}(v) - (n-n_1)\,q_2}{(n-n_1)(p_2-q_2)}, \qquad I_j^{\mathrm{II}}(v)=\sum_{i=n_1+1}^{n}\mathbb{I}(y_{i,j}=v). \] The final estimator combines the two phases by a weighted average: \[ \hat f_j(v)=\frac{n_1\,\hat f_j^{I}(v) + (n-n_1)\,\hat f_j^{\mathrm{II}}(v)}{n}. \] This preserves the usual unbiased GRR-style correction structure while allocating the noisy sample budget asymmetrically across users [2507.17516]. A substantial part of the mechanism is the choice of \(p_y\). The paper models the Phase-II mean squared error for a categorical value \(v\) as \[ MSE\bigl[\hat{f}^b_{1}(v)\bigr] = A(v)^2 + \frac{\tfrac{1}{4}-B(v)^2}{n'\Delta^2}, \] where \[ A(v)=\frac{d_0(v)}{2}+\frac{p_y e(v)}{2}, \qquad B(v)=\frac{\Delta}{2}\left[a_0(v)+p_y e(v)\right], \] \[ f_a(v)=\Pr[X_1=v], \qquad f_b(v)=\Pr[X_2=v], \] \[ d_0(v)=1-f_a(v)-f_b(v),\qquad e(v)=2f_b(v)-1, \] \[ p=\frac{e^\epsilon}{e^\epsilon+k-1}, \qquad q=\frac{1}{e^\epsilon+k-1}, \qquad \Delta=p-q. \] Averaging over all values yields a quadratic objective in \(p_y\), and the stated optimizer is \[ p_{y}^{*} = \frac{\displaystyle \sum_{v=0}^{d-1}\Bigl[ \frac{d_{0}(v)e(v)}{2d} -\frac{a_{0}(v)e(v)}{2n'd} \Bigr] } {\displaystyle \sum_{v=0}^{d-1}\Bigl[ \frac{e(v)^2}{4d} -\frac{e(v)^2}{4n'd} \Bigr] }. \] The endpoints \(p_y=0\) and \(p_y=1\) are also evaluated, and the best of \(\{0,p_y^*,1\}\) is chosen [2507.17516]. The privacy theorem is explicit: Corr-RR satisfies \(\epsilon\)-LDP. The proof uses three standard facts. Phase I is \(\epsilon\)-LDP by sequential composition because each user perturbs all \(d\) attributes with GRR using \(\epsilon/d\) per attribute. Phase II is \(\epsilon\)-LDP because one attribute is perturbed with budget \(\epsilon\) and the rest are produced by post-processing. Since the Phase-I and Phase-II user sets are disjoint, the combined mechanism remains \(\epsilon\)-LDP by parallel composition. The paper also notes a privacy amplification viewpoint for random attribute selection in Phase II, \[ \epsilon'=\ln\!\Bigl(\tfrac{1}{\beta}(e^\epsilon-1)+1\Bigr), \qquad \beta=\frac1d, \] while emphasizing that the formal guarantee remains \(\epsilon\)-LDP [2507.17516]. ## 5. Pairwise correlated randomized response: Joint RR A distinct but closely related correlated randomized response construction is Joint Randomized Response (JRR), which addresses binary frequency estimation under local differential privacy. JRR divides users into disjoint groups of two and introduces dependence between the two users’ perturbations while preserving the same one-user marginal behavior as classical RR. If pair \(G_i\) contains users \(u_{2i-1}\) and \(u_{2i}\), then the joint truthfulness vector \((T_{2i-1},T_{2i})\in\{0,1\}^2\) is sampled according to \[ \Pr[T_{2i-1}=1,T_{2i}=1]=p^2+\rho pq, \] \[ \Pr[T_{2i-1}=1,T_{2i}=0]=(1-\rho)pq, \] \[ \Pr[T_{2i-1}=0,T_{2i}=1]=(1-\rho)pq, \] \[ \Pr[T_{2i-1}=0,T_{2i}=0]=q^2+\rho pq, \] where \(q=1-p\), \(0.5