Pr²R: Privacy-Preserving Replay Techniques

• Pr²R is a design pattern that retains historical knowledge via compressed or synthetic representations instead of storing raw data.
• It is applied in federated learning, continual learning, person re-identification, face forgery detection, and secure process mining.
• Pr²R methods integrate current-data objectives with replay terms using techniques like DP-SGD and homomorphic encryption to meet diverse privacy standards.

Privacy-Preserving Replay (Pr$^2$R) denotes a family of methods that preserve previously acquired information while avoiding the storage, transmission, or direct reuse of raw historical data. Across the recent literature, the term appears in federated learning, continual learning, lifelong person re-identification, continual face forgery detection, and even secure process mining, but the common pattern is stable: past knowledge is reintroduced through compressed features, synthetic samples, latent prototypes, discrepancy maps, or encrypted intermediate states rather than through raw exemplars or centrally pooled datasets (Li et al., 31 Oct 2025, Pennisi et al., 2022, Harit et al., 28 Sep 2025, Wang et al., 3 Aug 2025, Sinhal et al., 11 Jan 2026, Rodríguez-Flores et al., 28 Apr 2026, Zhang et al., 14 Apr 2026).

1. Terminological scope and major instantiations

The label Pr$^2$R does not refer to a single algorithm. It is used for several architectures that share the same high-level objective—retaining historical knowledge under privacy constraints—but differ substantially in topology, replay carrier, and privacy model.

Setting	Replay carrier	Privacy basis
FedReplay (Li et al., 31 Oct 2025)	1 % of CLIP-extracted feature representations	non-reversible embeddings; optional Gaussian noise
FedER (Pennisi et al., 2022)	synthetic buffer $B_i$ from a conditional GAN	LPIPS-based privacy loss
EWC-guided diffusion replay (Harit et al., 28 Sep 2025)	class-conditional diffusion samples	no raw patient images stored
LReID Pr$^2$R (Wang et al., 3 Aug 2025)	pixel-space condensed samples plus style replay	face-ROI Gaussian blur; no raw images in memory
DP-FedEPC (Sinhal et al., 11 Jan 2026)	latent prototypes and logits	DP-SGD; Gaussian noise on shared summaries
Secure token-based replay (Rodríguez-Flores et al., 28 Apr 2026)	encrypted markings and transition encodings	homomorphic encryption
Direct Discrepancy Replay (Zhang et al., 14 Apr 2026)	distribution discrepancy maps	no raw historical faces; reduced linkability

In some works, replay is a learning primitive for mitigating catastrophic forgetting or non-IID drift. In others, replay is the central object of the computation itself, as in token-based conformance checking over encrypted traces. This suggests that Pr$^2$R is best understood as a design pattern: retain utility from the past without retaining the past in raw form.

2. Replay carriers and system architectures

A first architectural family uses feature-space replay. FedReplay couples a frozen CLIP vision transformer, specifically ViT-B/32, with a lightweight transformer-based classifier head of approximately 2 % of full parameters. Each client extracts embeddings $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$ with $d \approx 512$, performs per-class balanced sampling with $\alpha = 0.01$, uploads the resulting subset $R_i$, and participates in federated updates only through the compact classifier. The server forms a global replay buffer $\mathcal{R} = \cup_i R_i$, warm-start fine-tunes on $^2$0, and broadcasts both $^2$1 and $^2$2 during each round (Li et al., 31 Oct 2025).

A second family uses synthetic replay. FedER operates over a peer-to-peer federation of nodes without any central server. Each node trains a conditional GAN based on StyleGAN2-ADA, samples a synthetic buffer $^2$3, receives another node’s parameters and synthetic buffer, and fine-tunes on both real data and replayed synthetic data. The buffer size is fixed, for example 512, and is renewed each round (Pennisi et al., 2022). In exemplar-free continual medical imaging, Pr$^2$4R instead trains a class-conditional DDPM on each new task, generates replay samples for previous classes by ancestral sampling, and combines diffusion replay with Elastic Weight Consolidation; the classifier is a lightweight ViT and the generative model is a U-Net-style DDPM conditioned on class label $^2$5 (Harit et al., 28 Sep 2025).

A third family condenses information into compact image-like or latent carriers. In lifelong person re-identification, sequential data are condensed into the pixel space by matching gradients across multiple training snapshots $^2$6, producing a small synthetic set $^2$7 with $^2$8. A subsequent style-transfer network implements a dual-alignment strategy that stylizes current data toward previous domains and replay memory toward the current domain (Wang et al., 3 Aug 2025). In federated continual hospital imaging, DP-FedEPC stores only latent prototypes $^2$9, where $B_i$0 is an encoder embedding and $B_i$1 the corresponding logits, constructed through k-means over correctly classified examples (Sinhal et al., 11 Jan 2026). In continual face forgery detection, replay memory consists of a tiny bank of distribution discrepancy maps $B_i$2 rather than images (Zhang et al., 14 Apr 2026).

A fourth family uses encrypted state replay rather than data replay. In secure conformance checking, the model owner preprocesses the Petri net into matrices $B_i$3, $B_i$4, and $B_i$5, while the log owner repeatedly invokes an encrypted replay step over ciphertexts encoding the current marking and one-hot transition vector. The replay object is therefore an encrypted process state rather than a learned sample or feature (Rodríguez-Flores et al., 28 Apr 2026).

3. Optimization principles and mathematical structure

Despite the diversity of replay carriers, the optimization pattern is strikingly consistent: a current-data objective is augmented by a replay term, and many systems add a stabilization or privacy regularizer. In FedReplay, the client objective is

$B_i$6

with a global objective

$B_i$7

where $B_i$8 can enforce isotropy or add Gaussian noise to embeddings (Li et al., 31 Oct 2025).

FedER uses the same replay-versus-current-data decomposition at node level:

$B_i$9

Its privacy-aware generator objective is

$^2$0

where $^2$1 is an LPIPS-based pairwise privacy loss (Pennisi et al., 2022).

In continual medical imaging, diffusion replay is combined explicitly with synaptic consolidation:

$^2$2

The replay model is trained through the denoising score-matching objective, while EWC penalizes deviation from the previous optimum according to the diagonal empirical Fisher information (Harit et al., 28 Sep 2025).

In lifelong person re-identification, the condensation stage minimizes

$^2$3

where $^2$4 matches gradients between condensed and full data, and $^2$5 preserve identity discrimination. Training with style replay then uses

$^2$6

This formulation couples replay with domain adaptation rather than treating replay as a static memory mechanism (Wang et al., 3 Aug 2025).

DP-FedEPC adopts a closely related composite loss in federated continual learning:

$^2$7

Here replay is prototype-based, and regularization is global via a federated Fisher estimate (Sinhal et al., 11 Jan 2026).

Direct Discrepancy Replay moves replay from sample matching to discrepancy matching. It learns discrepancy maps by solving

$^2$8

where the discrepancy is measured through empirical characteristic functions in feature space. Replay is then generated through variance-preserving composition

$^2$9

with $^2$0 sampled from a truncated cosine schedule (Zhang et al., 14 Apr 2026).

In secure conformance checking, replay is not an auxiliary loss but the computation itself. The trace fitness is

$^2$1

and the encrypted replay step computes the next marking and missing-token count through matrix operations, comparisons, and masked selection on ciphertexts (Rodríguez-Flores et al., 28 Apr 2026).

4. Privacy models, guarantees, and threat assumptions

The privacy semantics of Pr$^2$2R vary substantially across the literature. FedReplay relies on the claim that CLIP embeddings are non-invertible: the embedding space is high dimensional, lossy, and entangles features nonlinearly, direct inversion methods fail without fine-tuned decoders, and empirical inversion attempts yield blurred, semantically incorrect reconstructions with low SSIM/PSNR. Privacy is further strengthened by sharing only 1 % of embeddings and, optionally, by adding Gaussian noise with small $^2$3 before sharing (Li et al., 31 Oct 2025). FedER similarly does not claim a formal DP bound; instead, it uses an LPIPS-based pairwise penalty to discourage GAN memorization, and privacy is quantified by latent-space backprojection distances, whose histograms shift away from zero when $^2$4 is active (Pennisi et al., 2022).

Exemplar-free continual medical imaging also emphasizes privacy through non-storage rather than formal privacy accounting: no raw patient images are stored, only DDPM parameters are kept, and synthetic samples bear no direct patient identifiers. The same work gives a theoretical bound connecting forgetting to the KL divergence between true and replay distributions and to Fisher-weighted parameter drift, thereby formalizing the fidelity–stability trade-off without stating a DP guarantee (Harit et al., 28 Sep 2025). In lifelong person re-identification, privacy is enforced by blurring the face-ROI of each real image before optimization and by subsequent pixel-level gradient updates that drive condensed images away from any single real image’s appearance; the stated consequence is that original images cannot be reconstructed from the synthetic memory (Wang et al., 3 Aug 2025).

DP-FedEPC is the clearest instance of formal privacy. It applies DP-SGD at each client through per-example gradient clipping at norm $^2$5 and Gaussian noise $^2$6, and it similarly clips and noises the Fisher vector and prototype summaries before transmission. Privacy is tracked with the Moments Accountant or Rényi DP accountant, and for $^2$7, $^2$8, $^2$9, and $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$0, the reported final $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$1 lies in the single-digit to low-teens range at $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$2 (Sinhal et al., 11 Jan 2026).

Secure conformance checking moves beyond statistical privacy and assumes a semi-honest adversary model. Confidentiality of traces follows because the client sends only encrypted one-hot transition vectors, and Paillier or TFHE-style Concrete encryption is used as the homomorphic substrate; confidentiality of the Petri net follows because the log owner sees only encrypted per-step results and never inspects the model matrices in the clear (Rodríguez-Flores et al., 28 Apr 2026). Direct Discrepancy Replay evaluates privacy at replay level through identity-level linkability and image-level similarity, reporting lower ArcFace linkage and lower replay-MIA AUC than selection-based replay baselines (Zhang et al., 14 Apr 2026).

A plausible implication is that Pr$$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$3R should not be read as a single privacy guarantee. Depending on the method, it may mean empirical obfuscation, exemplar-free retention, formal $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$4-DP, or cryptographic confidentiality.

5. Empirical behavior and reported gains

In smart agriculture, FedReplay is evaluated on CWD30, a 30-class dataset with 219 K images and 5 non-IID clients. The reported accuracies are 5.8 % for Raw-CLIP, 20.3 % for FedTPG, 52.8 % for StdFed without replay, 86.6 % for FedReplay, and 93.5 % for the centralized upper bound. FedReplay reaches 80 % accuracy in 100 rounds, whereas StdFed needs $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$5, and per-round transmission is reduced by 98 %. The replay-ratio ablation reports $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$6, $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$7, $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$8, and $$z_{i,j} = F_{\mathrm{CLIP}}(x_{i,j}) \in \mathbb{R}^d$$9, with diminishing returns thereafter; adding Gaussian noise with $d \approx 512$0 reduces inversion success to chance and drops accuracy by $d \approx 512$1 (Li et al., 31 Oct 2025).

In decentralized medical federation, FedER is tested on tuberculosis and melanoma classification. On tuberculosis, the reported mean accuracy is 83.41 % versus 77.82 % for FedAvg. Cross-node decision convergence is measured by the standard deviation across peers, with FedER yielding $d \approx 512$2 while standalone models show $d \approx 512$3. The privacy-enhanced variants remain competitive: FedER-A reaches 82.76 % versus 83.41 % for full FedER, and even FedER-B still matches or beats standard federated baselines (Pennisi et al., 2022).

In exemplar-free continual medical imaging, the CheXpert results are reported as follows: Finetune 64.8 % accuracy, 26.9 forgetting, and 0.802 AUC; EWC 70.5 %, 19.4, and 0.824; DER++ 73.2 %, 13.8, and 0.838; Pr$d \approx 512$4R 76.4 %, 10.9, and 0.851; Joint 79.1 %, 0.0, and 0.869. The method therefore reduces forgetting by over 30 % relative to DER++ while approaching the joint-training upper bound (Harit et al., 28 Sep 2025).

In federated continual hospital imaging, DP-FedEPC is evaluated on a simulated 4-site, 3-task CheXpert stream. The reported macro-AUROC and forgetting are 86.8 and 1.2 for static FedAvg, 80.4 and 9.7 for FedAvg-Seq, 85.0 and 3.8 for FedEWC, 84.3 and 4.1 for FedProto, 85.9 and 2.2 for DP-FedEPC with $d \approx 512$5, and 84.6 and 3.0 for DP-FedEPC with $d \approx 512$6. On external MIMIC-CXR validation, FedAvg-Seq obtains 78.4 % AUROC and DP-FedEPC with $d \approx 512$7 obtains 82.0 % (Sinhal et al., 11 Jan 2026).

In lifelong person re-identification, Pr$d \approx 512$8R is reported to yield roughly +4 pp mAP and +6 pp Rank-1 on seen domains relative to the best prior replay-based method, DASK. The concrete example given for Order 1, Seen-Avg is DASK $d \approx 512$9 mAP and 69.3 R@1 versus Pr$\alpha = 0.01$0R $\alpha = 0.01$1 mAP and 73.4 R@1. The paper also states that unseen-domain generalization remains on par with joint training (Wang et al., 3 Aug 2025).

In continual face forgery detection, DDC-MIR* reaches 91.08 % ACC with AF=4.34 % on Benchmark P1, compared with 90.08 % and 5.65 % for SUR-LID; DDC-MIR$\alpha = 0.01$2 reaches 92.16 % ACC with AF=2.85 %. On Benchmark P2, SUR-LID has final AUC=93.56 % and AF=1.26 %, DDC-MIR* has AUC=94.04 % and AF=2.84 %, and DDC-MIR$\alpha = 0.01$3 reaches AUC=95.23 % and AF=2.10 % (Zhang et al., 14 Apr 2026).

In encrypted process mining, the secure variants remain computationally practical but far slower than clear-text replay. For fitting traces of length 5, 9, and 13, SEC takes 7.8, 12.3, and 15.8 seconds and SEC$\alpha = 0.01$4 takes 19.3, 29.0, and 37.4 seconds; for unfitting traces of length 4, 8, and 12, SEC takes 6.1, 10.2, and 23.0 seconds and SEC$\alpha = 0.01$5 takes 17.0, 25.5, and 34.1 seconds. The reported end-to-end conformance time per trace under SEC$\alpha = 0.01$6 is under 1 minute (Rodríguez-Flores et al., 28 Apr 2026).

6. Misconceptions, limitations, and open problems

A common misconception is that replay necessarily means retaining old samples. The Pr$\alpha = 0.01$7R literature directly contradicts this. Replay may consist of feature subsets, synthetic GAN buffers, diffusion samples, latent prototypes, discrepancy maps, or encrypted state transitions, and several methods state explicitly that no raw images from past tasks are stored (Li et al., 31 Oct 2025, Harit et al., 28 Sep 2025, Wang et al., 3 Aug 2025, Sinhal et al., 11 Jan 2026, Zhang et al., 14 Apr 2026).

A second misconception is that “privacy-preserving” always implies a formal privacy theorem. This is not the case. FedER explicitly states that no formal DP bound is claimed, and its privacy evidence is empirical obfuscation through LPIPS distance histograms (Pennisi et al., 2022). FedReplay argues from CLIP non-invertibility, minimal attack surface, and optional Gaussian perturbation (Li et al., 31 Oct 2025). By contrast, DP-FedEPC provides formal $\alpha = 0.01$8-DP through DP-SGD (Sinhal et al., 11 Jan 2026), and secure token-based replay relies on cryptographic confidentiality under a semi-honest model rather than on statistical indistinguishability alone (Rodríguez-Flores et al., 28 Apr 2026).

Several limitations recur. Diffusion replay is compute-intensive, and unified DDPMs across all tasks can save memory at slight fidelity cost; real-world streams may also involve class imbalance, evolving taxonomies, and richer modalities (Harit et al., 28 Sep 2025). Direct Discrepancy Replay assumes that current real-face carriers remain semantically valid canvases for previous-task discrepancy cues, and it notes that replay fidelity may degrade if the face manifold shifts drastically (Zhang et al., 14 Apr 2026). Secure conformance checking currently handles safe Petri nets and explicit silent transitions, clamps markings to $\alpha = 0.01$9 thereby losing flooding diagnostics, and would require zero-knowledge proofs to move from semi-honest to malicious adversaries (Rodríguez-Flores et al., 28 Apr 2026). In federated continual learning, formalizing a rigidity-plasticity trade-off under DP noise remains an open theoretical direction (Sinhal et al., 11 Jan 2026).

Taken together, these limitations indicate that Pr$R_i$0R is not a settled technique but an evolving methodological cluster. What unifies the cluster is not a single algorithmic recipe, but a shared operational principle: recover the benefits of replay while replacing raw historical data with a privacy-compatible surrogate that still carries enough information to preserve performance under distribution shift, non-IID federation, or continual task evolution.