Compressed Aggregate Feedback (CAFe)

• CAFe is a communication-efficient feedback scheme that aggregates compressed client updates relative to a shared predictor, ideal for distributed and federated learning.
• It leverages techniques such as compressive sensing and error-feedback to reconstruct full updates, ensuring convergence and reducing uplink communication dramatically.
• Empirical results show CAFe improves test accuracy and feedback performance in systems like MIMO and deep learning, balancing compression aggressiveness with reconstruction guarantees.

Compressed Aggregate Feedback (CAFe) is a class of communication-efficient feedback schemes that leverage aggregation, compression, and, where relevant, compressive sensing or error-feedback constructs to drastically reduce uplink (or feedback) overhead in distributed, wireless, and federated learning systems. The core design principle is to aggregate feedback or updates in a compressed domain, often with respect to a shared predictor such as a previous global aggregate, a server-guided update, or a compressed sensing basis. CAFe frameworks are rigorously analyzed in the context of distributed optimization, MIMO channel state feedback, and large-scale deep learning, providing sharp theoretical guarantees and considerable empirical savings.

1. Foundational Methodologies of Compressed Aggregate Feedback

CAFe denotes feedback architectures where clients (or users) communicate a compressed update—often a difference from a shared aggregate or structured predictor—instead of direct, full-dimensional information. Foundational implementations appear in multiple domains:

• Distributed and Federated Learning: Clients transmit the compressed difference between their local update and the prior round’s global aggregated update (or a server-guided predictor). The server reconstructs full updates by summing received compressed differences with the shared predictor (Ortega et al., 2024, Ortega et al., 27 Dec 2025).
• MIMO Feedback: Users with strong channel gains transmit feedback using shared channels and unique signatures. The base station aggregates all signals and decodes both user identities and their feedback values via compressive sensing (Qaseem et al., 2010, Lee et al., 2014).
• Preconditioned Optimization: In deep learning, gradients are sparsified or projected before entering the preconditioner memory window, and the error from compression is fed back into future steps (error-feedback). The sliding window of compressed gradients drives full-matrix preconditioning with drastically reduced memory (Modoranu et al., 2023).

Across these applications, the unifying elements are (1) feedback or update compression against a global, aggregate, or predictive baseline, and (2) recovery of the aggregate effect at the receiver via decoding, error-feedback, or compressive sensing.

2. CAFe Algorithms and Update Equations

A representative template for CAFe in distributed optimization is as follows:

1. At round $k$, the server broadcasts the current model $x^k$ and the previous aggregate update $\Delta_s^{k-1}$ (initially zero).
2. Each client $n$ computes its local update $\Delta_n^k = -\gamma \nabla f_n(x^k)$.
3. The client compresses the offset $\Delta_n^k - \Delta_s^{k-1}$ with a biased or unbiased compression operator $\mathcal Q$ to obtain $\mathcal Q(\Delta_n^k - \Delta_s^{k-1})$.
4. The server reconstructs each client’s pseudo-update as $$\hat\Delta_n^k = \mathcal Q(\Delta_n^k - \Delta_s^{k-1}) + \Delta_s^{k-1}$$.
5. The new global aggregate is $$\Delta_s^k = \frac{1}{N} \sum_{n=1}^N \hat\Delta_n^k$$, and the global model is updated as $x^{k+1} = x^k + \Delta_s^k$ (Ortega et al., 27 Dec 2025, Ortega et al., 2024).

In the context of compressive sensing for MIMO feedback, the base station receives observations stacked as $\mathbf{y} = \mathbf{\Phi s} + \mathbf{n}$, where $\mathbf{s}$ is a sparse vector of active users’ feedback; $\mathbf{y}$ is decoded using standard sparse recovery algorithms such as $\ell_1$-minimization or Orthogonal Matching Pursuit (Qaseem et al., 2010).

For compressed preconditioning, each incoming gradient $g_t$ is replaced by its compressed version $c_t = \mathrm{Compress}(a_t)$ with $a_t$ including the current gradient and the error buffer $\xi_{t-1}$. The error feedback is $\xi_t = a_t - c_t$, and this procedure ensures that all gradient components are eventually included in the memory (Modoranu et al., 2023).

3. Theoretical Guarantees and Error Analysis

CAFe architectures admit comprehensive theoretical analyses:

• Convergence Rate: In distributed gradient descent (DGD) with CAFe and biased compression (parameter $\omega < 1$), the average squared gradient norm over $K$ rounds is bounded by

$$\frac{1}{K}\sum_{k=0}^{K-1} \mathbb{E}\left\|\nabla f(x^k)\right\|^2 \leq \frac{2(f(x^0)-f^*)}{\gamma K} \cdot \frac{1-\omega}{1-\omega B^2}$$

for step size $$\gamma \leq \frac{1-\omega}{L(1+\omega)},\ \omega B^2 < 1$$ with $B^2$ bounding gradient dissimilarity (Ortega et al., 27 Dec 2025, Ortega et al., 2024). This gives an explicit $(1-\omega)$ acceleration factor compared to direct compression (DCGD).

• Compressive Sensing Recovery: In feedback reduction for MIMO, if the number of measurements $M$ satisfies $M \geq C K \log(N/K)$ (with $K$ the number of strong users and $N$ total users), perfect or robust recovery of the sparse vector is guaranteed by the Restricted Isometry Property, using known CS solvers (Qaseem et al., 2010).
• Error-Feedback: For preconditioners, error-feedback (EF) applied to compressed gradients ensures that the total error is bounded and does not impact asymptotic convergence. Application of Top-$k$ or low-rank compression, combined with EF, recovers both the convergence and accuracy benefits of dense full-matrix preconditioners (Modoranu et al., 2023).

The proofs utilize smoothness and Lyapunov drift arguments, coupled with compression-induced error recursions specific to the CAFe update structure.

4. Applications and Domain-Specific Instantiations

Distributed and Federated Learning: CAFe is used to efficiently compress uplink client-to-server updates in federated optimization, eliminating the need for client-specific control variates and thus supporting stateless, privacy-preserving clients. Empirically, CAFe with Top-$k$, quantized, or SVD compression matches or outperforms direct compression under aggressive regimes, especially in heterogeneous or non-iid client scenarios (Ortega et al., 2024, Ortega et al., 27 Dec 2025).

MIMO Feedback Systems: CAFe architectures have been deployed in both analog and digital feedback channels. In the analog variant, the joint recovery reduces effective noise variance as $\sigma^2 / M$; in digital, quantized SNR values are packed via compressive sensing, reducing feedback dimensions from $O(N)$ (dedicated per-user) to $O(\log N)$ with near-dedicated sum-rate performance (Qaseem et al., 2010). Antenna group-based CAFe realizes further compression by mapping correlated elements to low-dimensional aggregates, followed by structured quantization and expansion (Lee et al., 2014).

Full-Matrix Preconditioning in Deep Learning: In the EFCP instantiation, CAFe enables memory and compute savings (up to $60\times$ reduction) in sliding-window–based preconditioners such as M-FAC or GGT, with no loss in final accuracy or convergence epochs on large-scale vision and language tasks (Modoranu et al., 2023).

5. Empirical Performance and Trade-Offs

CAFe frameworks consistently deliver significant efficiency gains:

• Federated/DGD Setup: On datasets such as MNIST, EMNIST, and CIFAR (10/100), CAFe achieves up to 10% higher test accuracy over direct compression at extreme sparsity (e.g., SVD-rank 1, non-iid splits), and recovers nearly full accuracy when direct compression fails ($\leq$13% for direct, $\sim$72.5% for CAFe on CIFAR-10 Top-1%-4-bit) (Ortega et al., 2024, Ortega et al., 27 Dec 2025).
• Feedback Overhead in Wireless: In MIMO, CAFe reduces feedback from $O(N)$ per user to $O(\log N)$ shared dimensions, while maintaining a vanishing sum-rate gap to dedicated feedback as $N\to\infty$. In FDD massive MIMO, antenna grouping and CAFe realize 50–70% bit savings vs. full vector quantization at the same sum-rate, and require only $18$ rather than $32$ bits per user to achieve a given throughput (Lee et al., 2014, Qaseem et al., 2010).
• Preconditioning in Deep Networks: On modern workloads (ViT-Tiny, BERT, ResNet-18), S-M-FAC (Top-1% CAFe) recovers dense method accuracy with $30$--$60\times$ reduced memory; low-rank methods perform similarly, verifying the EF+compression approach (Modoranu et al., 2023).

A central trade-off is between the compression aggressiveness (controlled by $\omega$ for client updates, dimension $M$ in MIMO, or rank/sparsity $k$ in preconditioning) and the convergence rate or residual error. CAFe admits tunable parameters (sparsity, grouping design, predictor source—aggregate or server-guided) allowing adaptation to task and system constraints.

6. Extensions: Server-Guided Predictors and Generalizations

Server-Guided Compressed Aggregate Feedback (CAFe-S) generalizes CAFe for scenarios where the server holds a small proxy dataset. Here, clients compress their update with respect to a server-generated predictor $\Delta_c^k = -\gamma \nabla f_s(x^k)$. If the server dataset is representative ($G^2$ small), the convergence rate further improves to

$$\frac{1}{K} \sum_{k=0}^{K-1} \mathbb{E}\|\nabla f(x^k)\|^2 \leq \frac{2(f(x^0)-f^*)}{\gamma K (1-\omega G^2 B^2)}$$

demonstrating enhanced performance as server–client similarity increases (Ortega et al., 27 Dec 2025).

Other generalizations include compressive aggregation in block-fading channels, aggregate feedback for massive MIMO via grouping and codebook methods, and the possibility of exploiting group or block sparsity in the aggregate signal for further gains (Qaseem et al., 2010, Lee et al., 2014).

7. Context, Related Protocols, and Implications

CAFe challenges the previous paradigm of client-local error-feedback and control variates, establishing that stateless, global predictors (such as the last aggregate or a server-guided update) suffice for biased compression with provable acceleration and practical benefits. Unlike direct compression (DCGD), CAFe achieves strict improvement proportional to the compression bias parameter $(1-\omega)$. It generalizes to models where privacy, scalability, or system heterogeneity preclude client state. In wireless and sensing regimes, CAFe fuses compressive-sensing recovery with opportunistic access to maintain near-optimal information extraction with exponentially reduced overhead.

A plausible implication is that, as systems scale and communication costs dominate, CAFe-style aggregate and predictor-based feedback architectures will become standard for high-dimensional, bandwidth-constrained distributed learning, as well as for high-user-count wireless feedback and large-batch deep network training.

---

References

• (Qaseem et al., 2010) "Compressed Sensing for Feedback Reduction in MIMO Broadcast Channels"
• (Lee et al., 2014) "Antenna Grouping based Feedback Compression for FDD-based Massive MIMO Systems"
• (Modoranu et al., 2023) "Error Feedback Can Accurately Compress Preconditioners"
• (Ortega et al., 2024) "Communication Compression for Distributed Learning without Control Variates"
• (Ortega et al., 27 Dec 2025) "Communication Compression for Distributed Learning with Aggregate and Server-Guided Feedback"