Federated Continual Recommendation (FCRec)

• FCRec is a paradigm combining federated learning and continual recommendation to handle evolving user-item interactions while ensuring data privacy.
• The framework employs an adaptive client-side replay memory that mitigates catastrophic forgetting by selectively retaining past user preferences based on measured shifts.
• A server-side item-wise temporal aggregation mechanism balances new updates with historical embeddings, enhancing model stability and adaptation efficiency.

Federated Continual Recommendation (FCRec) is a paradigm that combines the privacy-preserving principles of federated learning (FL) with the adaptive, non-stationary modeling demands of continual recommendation (CLRec). FCRec formalizes the task of streaming, privacy-sensitive collaborative filtering in which models must continually adapt to evolving user–item interaction data without centralized access to raw behavioral records. This approach addresses stringent privacy, adaptability, and efficiency requirements unmanageable by traditional federated or continual learning methods alone (Lim et al., 6 Aug 2025).

1. Motivation and Problem Formulation

FCRec is motivated by two converging trends: (1) the widespread enforcement of data privacy via policies that preclude centralized collection of user data, and (2) the observation that user preferences and item catalogues naturally evolve, resulting in pronounced non-stationarity in the data streams underpinning recommendation systems. While FL protects privacy by distributing computation such that only parameter updates are shared, most federated recommendation (FedRec) research assumes a fixed or stationary data distribution. Contrarily, CLRec methods excel at adapting to non-stationary, streaming data but typically require centralized data access for replay, distillation, or regularization mechanisms, violating federated privacy constraints.

The FCRec problem is defined as learning recommendation models in a setting where each client observes a local, temporally partitioned stream of user–item interactions. The system must reconcile two core objectives:

• Knowledge Retention: Preventing catastrophic forgetting of users' earlier preferences as new data arrives.
• Adaptation: Rapidly integrating new behavioral trends and item information, all while raw data remains strictly localized to each client.

A key challenge is “privacy-aware continual adaptation”: enabling both effective knowledge transfer and temporal learning at scale without compromising user confidentiality (Lim et al., 6 Aug 2025).

2. Challenges in Federated Continual Recommendation

FCRec must simultaneously address a compound set of challenges:

• Non-Stationary Data Streams: Continuous drifts in user interests and the dynamic introduction of new items undermine the efficacy of federated models trained on static snapshots. Without mechanisms to track such changes, model accuracy degrades over time.
• Forgetting and Interference: The classical continual learning problems of “forgetting” (loss of valuable past knowledge) and “interference” (instability from conflicting old and new knowledge) are exacerbated in the federated setting due to the lack of global replay memory or direct central access to user histories.
• Stringent Privacy Constraints: Federated protocols strictly forbid centralized collection/storage of private user data, thus precluding CLRec solutions based on global model rehearsal or batch assimilation of interaction events.
• Heterogeneous Update Patterns: As user behaviors shift asynchronously and client participation is intermittent, server-side aggregation functions must avoid over-emphasizing volatile local updates or disregarding slowly evolving client histories.
• Scalability and Efficiency: The need to adapt in real time on bandwidth-limited devices, with thousands to millions of participants, imposes high communication and computation efficiency requirements.

These constraints preclude naive extensions of existing CLRec or FedRec algorithms, requiring new algorithmic design (Lim et al., 6 Aug 2025).

3. The F³CRec Framework: Client- and Server-side Continual Learning

F³CRec is a modular framework designed to achieve continual, privacy-preserving adaptation in federated settings, introducing complementary strategies at client and server levels.

3.1 Client-Side: Adaptive Replay Memory

Each client maintains an Adaptive Replay Memory that selectively retains a proportion of past preferences based on measured user-specific preference shifts:

• At time $t$, after receiving a new local data block $D^t$, the client initializes its private parameters with those from the previous block $D^{t-1}$.
• The user’s historical Top-$N$ item list $S_u^{(t{-}1)}$ is compared to the ranking induced by the current model, yielding a preference shift metric:

$$\Delta_u^{(t,r)} = \sum_{i \in S_u^{(t{-}1)}} |r_u^{(t,r)}(i) - r_u^{(t{-}1)}(i)|$$

where $r_u^{(t,r)}(i)$ is the rank of item $i$ in the latest local model.

• An exponential decay function converts this shift into an adaptive sampling rate:

$$\delta_u^{(t,r)} = \exp(-\varepsilon \cdot \Delta_u^{(t,r)})$$

with hyperparameter $\varepsilon$.

• The client samples (without replacement) $\lceil\delta_u^{(t,r)}\ \cdot|S_u^{(t{-}1)}|\rceil$ items from $S_u^{(t{-}1)}$ to form the replay memory $M_u^{(t,r)}$.
• A knowledge distillation loss is imposed between the outputs of the “teacher” (previous model) and the current “student” model on $M_u^{(t,r)}$:

$$L_{\mathrm{KD}}^{(t,r)}(u) = \sum_{i \in M_u^{(t,r)}} [\hat{y}_u^{(t{-}1)}(i) \log \hat{y}_u^{(t,r)}(i) + (1{-}\hat{y}_u^{(t{-}1)}(i)) \log (1{-}\hat{y}_u^{(t,r)}(i))]$$

• The total loss is $$L_{\mathrm{Rec}}^{(t,r)}(u) + \lambda_{KD} L_{\mathrm{KD}}^{(t,r)}(u)$$.

This mechanism enables each user to automatically adjust the extent of historical knowledge retention: users with stable preferences (small $\Delta_u$) retain more past information, while users experiencing large behavioral shifts replay only a small fraction of historical samples, thus prioritizing adaptation (Lim et al., 6 Aug 2025).

3.2 Server-Side: Item-wise Temporal Mean Aggregation

The server maintains only public parameters (typically, item embeddings). Rather than simply averaging user uploads, it employs an item-wise temporally weighted mean that adaptively mixes new and old knowledge for each item:

• After each round $r$, denote the averaged update by $Q_g'^{(t,r)}$, the previous global state $Q_g^{(t-1)}$, and the item embedding dimension $d$.
• For each item $i$, compute a knowledge shift:

$$\phi_i^{(t,r)} = 1 / (\sqrt{d} \cdot \| Q_{g,i}^{(t-1)} - Q_{g',i}^{(t,r)} \|_2^2)$$

• Define an adaptive retention coefficient:

$\gamma_i^{(t,r)} = \beta / (1 + \phi_i^{(t,r)})$

with hyperparameter $\beta$.

• Update each item embedding as:

$$Q_{g,i}^{(t,r+1)} = (1 - \gamma_i^{(t,r)}) Q_{g',i}^{(t,r)} + \gamma_i^{(t,r)} Q_{g,i}^{(t-1)}$$

Items with large embedding shifts receive more weight on the fresh average; those with minimal change are more strongly regularized toward historical representations, balancing stability and plasticity across the catalog (Lim et al., 6 Aug 2025).

4. Experimental Evaluation and Comparative Analysis

Experiments were conducted on four real-world datasets (e.g., ML-100K, ML-Latest-Small, Lastfm-2K, HetRec2011) with non-stationary (streaming) interactions split into sequential “blocks” simulating real-time evolution. F³CRec was instantiated atop three prominent federated recommendation backbones: FedMF (matrix factorization), FedNCF (neural collaborative filtering), and PFedRec (personalized federated recommendation).

Comparative methods included:

• Fine-tuning and regularization-based continual learning baselines;
• Fixed-replay strategies without adaptation;
• Knowledge distillation schemes without user-specific or item-specific adaptation.

Key findings:

• F³CRec achieved significant and consistent improvements in time-aware recommendation quality, with NDCG@20 and Recall@20 increases often exceeding 20% in challenging setups.
• Ablation studies demonstrated that both the adaptive replay memory and item-wise temporal mean are indispensable for maintaining accuracy under concept drift; replacing either by static or naive strategies led to marked performance drops.
• The approach exhibited robust knowledge retention as evidenced by mitigated forgetting for both user- and item-level histories, while also facilitating effective adaptation to emerging behaviors.

These results establish F³CRec as an effective solution for FCRec, outperforming existing methods when subjected to realistic, temporally non-stationary federated recommendation scenarios (Lim et al., 6 Aug 2025).

5. Theoretical Foundations and Key Algorithms

Mathematical underpinnings for F³CRec are grounded in adaptive knowledge distillation and temporal aggregation. The essential constructs at the client level are:

• Preference Shift:

$$\Delta_u^{(t,r)} = \sum_{i \in S_u^{(t-1)}} |r_u^{(t,r)}(i) - r_u^{(t-1)}(i)|$$

• Adaptive Replay Memory Sampling:

$$\delta_u^{(t,r)} = \exp(-\varepsilon \cdot \Delta_u^{(t,r)})$$

• Knowledge Distillation Loss:

$$L_{\mathrm{KD}}^{(t,r)}(u) = \sum_{i \in M_u^{(t,r)}} [\hat{y}_u^{(t-1)}(i) \log \hat{y}_u^{(t,r)}(i) + (1{-}\hat{y}_u^{(t-1)}(i)) \log (1{-}\hat{y}_u^{(t,r)}(i))]$$

On the server:

• Item-wise Knowledge Shift and Retention:

$$\phi_i^{(t,r)} = 1/(\sqrt{d} \cdot \|Q_{g,i}^{(t-1)} - Q_{g',i}^{(t,r)}\|_2^2)$$

$\gamma_i^{(t,r)} = \beta/(1 + \phi_i^{(t,r)})$

$$Q_{g,i}^{(t,r+1)} = (1 - \gamma_i^{(t,r)})Q_{g',i}^{(t,r)} + \gamma_i^{(t,r)}Q_{g,i}^{(t-1)}$$

These mechanisms preserve the privacy-locality constraint—no user data leaves the client—while facilitating continual adaptation. No historical data or representations are stored server-side other than temporal summaries of public parameters.

6. Future Directions and Open Problems

F³CRec points to several promising directions for further research:

• Formal Privacy Guarantees: While the framework ensures privacy by restricting exchanges to public parameters (with optional stochastic noise), the development of formal local differential privacy mechanisms or integration of secure aggregation protocols (e.g., homomorphic encryption, secure multiparty computation) is needed to offer robust, provable protections.
• Enhanced Adaptivity: Improving user preference shift estimation, adaptive sampling functions, and more nuanced item-wise aggregation could increase robustness in highly dynamic environments.
• Scalability and System Heterogeneity: Extending FCRec to ultra-large, heterogeneous populations and asynchronous client participation, including stragglers and intermittent device availability.
• Integration with Meta-learning and Graph-based Methods: Combining FCRec with meta-learning, attention-based, or graph neural network personalization could enhance adaptation, especially in cold-start or multi-domain contexts.
• Real-world Deployment: Moving beyond static benchmarks to evaluate in deployment-scale, event-driven FCRec with real-world communication, dropouts, and adversarial conditions remains an open application area.

A plausible implication is that embedding these continual adaptation principles into general privacy-preserving recommendation protocols may become standard for regulatory-compliant, high-utility personalization in the streaming era (Lim et al., 6 Aug 2025).

---

In summary, Federated Continual Recommendation formalizes the challenge of temporally adaptive collaborative filtering under federated privacy constraints. The F³CRec framework offers a dual mechanism—adaptive client-side knowledge replay and temporally smoothed server-side parameter aggregation—shown to be effective under non-stationary real-world data streams. This task opens new research avenues at the intersection of continual learning, federated optimization, and privacy-aware personalization.