DP-KSA: Differentially Private Retrieval-Augmented QA

Updated 5 July 2026

DP-KSA is a privacy-preserving method that reformulates answer generation into a private keyword-selection problem instead of full-text synthesis.
The algorithm employs retrieval of relevant documents, ensemble generation of candidate answers, and adaptive top-k selection with propose-test-release to ensure differential privacy.
Empirical evaluations on benchmarks like Natural Questions and TriviaQA demonstrate a privacy-utility tradeoff, with robust performance under moderate privacy budgets.

DP-KSA is a differentially private retrieval-augmented generation (DP-RAG) algorithm for question answering over sensitive corpora. It is designed for settings in which retrieval improves factuality, but the retrieved database may contain private material whose contents could be exposed through model outputs. The method, introduced in "Differentially Private Retrieval-Augmented Generation" (Tang et al., 16 Feb 2026), avoids privatizing full autoregressive text generation token by token. Instead, it transforms answer generation into a private keyword-selection problem: it retrieves multiple contexts, generates an ensemble of candidate responses, privately extracts recurrent keywords, and conditions final generation only on the query plus those differentially private keywords.

1. Definition and problem setting

DP-KSA addresses a specific privacy failure mode of RAG systems. In the black-box threat model considered by the paper, an adversary can submit prompts and inspect outputs, while the private corpus $D$ used for retrieval is not directly exposed. Nevertheless, because standard RAG injects retrieved passages into the prompt, the generator can regurgitate sensitive context. The paper explicitly frames this as a problem of preventing leakage from corpora such as medical records or legal documents while retaining the utility benefits of retrieval (Tang et al., 16 Feb 2026).

The central observation underlying DP-KSA is that many QA answers are short and can be represented by a few key tokens. The paper reports that on Natural Questions and TriviaQA, most ground-truth answers are 1–4 tokens long. This motivates a compression strategy: rather than privately generating a full sentence over a vocabulary that can be around 50k tokens, DP-KSA privately identifies a compact set of answer-bearing keywords and then uses those keywords as the only retrieval-derived signal in the final prompt.

This design changes the privacy locus of the system. Privacy protection is applied to a small histogram over candidate-answer tokens rather than to every decoding step. A plausible implication is that the method is structurally aligned with extractive or short-answer QA, where the semantic content can often be preserved despite aggressive dimensionality reduction.

2. Algorithmic pipeline

DP-KSA has three main stages: retrieval, ensemble generation, and private keyword release. The retrieval component $R$ returns the top- $N$ relevant documents from the private database for a query $\mathbf{x}$ . Instead of concatenating all retrieved documents into one prompt, the generator $F$ is applied separately to each retrieved document together with the query, producing an ensemble of responses $\{y_i\}_{i=1}^N$ . The system then builds a token-frequency histogram over that ensemble and uses private mechanisms to determine how many keywords to release and which keywords to keep (Tang et al., 16 Feb 2026).

Stage	Operation	Output
Retrieval	Fetch top- $N$ relevant documents from private database	Retrieved contexts
Ensemble generation	Generate one response per query-context pair	Candidate answers $\{y_i\}_{i=1}^N$
Private summarization	Build token histogram, choose $\hat{k}$ , release top- $\hat{k}$ keywords	DP keywords for final prompt

The ensemble is not merely a redundancy device. The paper treats repeated tokens across independently conditioned candidate responses as a consensus signal: if a token is genuinely supported by the query and corpus, it should recur; if it is idiosyncratic or leakage-prone, it is less likely to appear consistently. DP-KSA therefore uses frequency stability as a proxy for semantic robustness.

The final generation step uses only the query and the released keywords. Crucially, the final prompt does not include the retrieved private documents directly. This matters because the last generation stage is then post-processing of a DP output, so it incurs no additional privacy loss. Conceptually, the method inserts a low-dimensional semantic interface between retrieval and generation.

3. Private keyword selection and formal guarantees

The private core of DP-KSA combines the propose-test-release (PTR) paradigm with adaptive top- $R$ 0 selection. Let $R$ 1 denote the token histogram induced by the ensemble of candidate responses, and let $R$ 2 be the $R$ 3-th largest count. The paper defines the gap

$R$ 4

This gap measures the stability of the top- $R$ 5 boundary (Tang et al., 16 Feb 2026).

The first private subroutine, FindBestK, chooses the number of keywords adaptively by maximizing a noisy score:

$R$ 6

Here $R$ 7 is a dataset-independent regularizer, and the Gumbel perturbation implements the exponential mechanism. The paper states that $R$ 8, so this step admits a standard DP analysis through sensitivity of the utility function.

The second subroutine, TopKWithPTR, determines whether the top- $R$ 9 tokens are locally stable enough to be released exactly. It uses the noisy test statistic

$N$ 0

If $N$ 1, the algorithm returns the exact top- $N$ 2 tokens; otherwise it terminates with no keywords. The intuition, stated explicitly in the paper, is that if the gap between the $N$ 3-th and $N$ 4-th tokens exceeds 2, then changing one document cannot alter the identity of the top- $N$ 5 set, so the local sensitivity of that output is zero.

The formal privacy target is standard $N$ 6-DP:

$N$ 7

for neighboring datasets $N$ 8 and $N$ 9 differing in one document. The paper proves that DP-KSA satisfies $\mathbf{x}$ 0-DP with respect to the private RAG database. The proof proceeds via Rényi differential privacy: the appendix establishes privacy bounds for FindBestK and TopKWithPTR, composes them in RDP, and then converts back to standard approximate DP (Tang et al., 16 Feb 2026).

A common misconception is that DP-KSA privatizes the final LLM itself. More precisely, the method privatizes the keyword release; the final textual answer is produced by ordinary generation applied as post-processing to the DP keywords. The formal guarantee therefore concerns the generated output through the privacy of the intermediate released keywords.

4. Empirical evaluation and privacy-utility behavior

The empirical study evaluates DP-KSA on two open-domain QA benchmarks, Natural Questions (NQ) and TriviaQA (TQA), using subsets of 100 questions from each dataset after filtering out examples with empty references. Retrieval is performed over Wikipedia using DPR, with FAISS for acceleration and inner-product similarity. Three instruction-tuned generators are tested: Qwen 2.5 (3B), Llama 3.2 (3B), and Llama 3.1 (8B). The privacy parameters are $\mathbf{x}$ 1 and $\mathbf{x}$ 2. Reported metrics are F1, ROUGE-1, ROUGE-L, and normalized Levenshtein similarity (Tang et al., 16 Feb 2026).

The paper compares DP-KSA against three reference points: Non-RAG $\mathbf{x}$ 3, Non-private KSA $\mathbf{x}$ 4, and Non-private RAG $\mathbf{x}$ 5. The main empirical pattern is a clear privacy-utility tradeoff: performance generally improves as $\mathbf{x}$ 6 increases, reflecting reduced noise and a higher PTR pass rate. At moderate privacy budgets, DP-KSA consistently outperforms the non-RAG baseline, showing that the released keywords preserve useful retrieval information.

Several more specific findings are emphasized. First, DP-KSA often performs similarly to, and sometimes better than, the non-private keyword baseline, suggesting that ensemble consensus can survive the added noise. Second, stronger generators benefit more from the method; the 8B Llama model is more robust than the smaller models when conditioned on sparse keyword prompts. Third, TriviaQA is easier than Natural Questions for DP-KSA, which the paper attributes to TQA being more extractive and redundant. Fourth, increasing the number of ensembles helps up to a point—performance improves from 10 to about 80 ensembles, then plateaus or slightly degrades because less relevant documents add redundancy and noise (Tang et al., 16 Feb 2026).

These results situate DP-KSA between two extremes. It does not match full non-private RAG in all settings, because keyword compression necessarily discards some contextual information. But it improves substantially over zero-retrieval generation while providing a formal privacy guarantee that ordinary RAG does not provide.

5. Position within the literature and nomenclatural ambiguity

DP-KSA belongs to the emerging literature on privacy-preserving RAG. Its main distinction, relative to approaches that inject DP into token-by-token generation, is architectural rather than merely parametric. The paper argues that naive DP at each decoding step is poorly suited to autoregressive generation because the response space is combinatorially large and utility degrades rapidly. DP-KSA instead moves privacy control into a smaller keyword-selection problem, using retrieval ensembles, adaptive top- $\mathbf{x}$ 7 selection, PTR, and post-processing generation (Tang et al., 16 Feb 2026).

The label "DP-KSA" is potentially ambiguous across arXiv. In other contexts, "KSA" denotes the Kingdom of Saudi Arabia, as in work on e-learning impact measurement and online retailing growth [(Alkhalaf et al., 2013); (Alghamdi et al., 2012)]. Likewise, "DP" may refer to DP-coloring in graph theory (Kostochka et al., 2023) or dp-rank in model theory (Kaplan et al., 2012). In current RAG literature, however, DP-KSA refers specifically to the differentially private keyword-based summarization and answer-generation scheme proposed in (Tang et al., 16 Feb 2026).

This ambiguity matters because the acronym itself does not transparently encode the method’s full scope. Within the DP-RAG setting, the defining feature is not merely the use of differential privacy, but the combination of ensemble response aggregation and private release of stable keywords.

6. Limitations, failure modes, and prospective extensions

The paper identifies several limitations. If the PTR test fails, no keywords are released, and the system can fall back toward non-RAG behavior. At low privacy budgets such as $\mathbf{x}$ 8, this occurs more often because the noise is larger and the top- $\mathbf{x}$ 9 boundary is harder to certify. Small models also struggle with sparse keyword prompts; the paper notes that the smallest Qwen model sometimes performs better in non-RAG mode than with keyword-based prompting. In addition, non-private KSA remains below full non-private RAG in many settings, indicating that keyword compression does not preserve all answer-relevant information (Tang et al., 16 Feb 2026).

These limitations delineate the regime in which DP-KSA is most appropriate. The method is especially well matched to short-answer QA, extractive benchmarks, and settings in which releasing a compact semantic sketch is sufficient for the downstream generator. A plausible implication is that tasks requiring long-form synthesis, nuanced reasoning over multiple passages, or stylistically constrained outputs may require richer private intermediates than a top- $F$ 0 keyword set.

The paper also points to future work on adaptively selecting the number of ensembles. More generally, the design suggests a broader research direction: replace direct private generation from sensitive retrieval contexts with private release of low-dimensional, task-aligned summaries. In DP-KSA, that summary is a set of stable keywords; the broader methodological contribution is the claim that semantic compression can be the key interface between retrieval utility and formal privacy.