Gradient-Free Aggregation: Paradigm Shift

• Gradient-Free Aggregation is a method that eliminates gradient communication by leveraging likelihood-free Bayesian inference in federated learning and deterministic gradient flows in growth models.
• It employs local SuffiAE encoders to compute private summaries, thereby reducing exposure to gradient-based privacy attacks while maintaining model accuracy.
• In stochastic growth models, GFA uses deterministic rules based on power-law interactions to form clusters with predictable structural and scaling properties.

Gradient-Free Aggregation (GFA) refers to a class of aggregation or learning methods in which gradients—either of loss functions or aggregation energies—are not communicated between distributed agents or particles. Two distinct paradigms introducing "gradient-free" aggregation mechanisms have emerged: the first in privacy-preserving federated learning via likelihood-free Bayesian inference (Hahn et al., 2020), and the second in stochastic geometric growth models via deterministic gradient-flow rules without stochastic path integration (Steinerberger, 2023). Both approaches fundamentally replace or obviate the need for classic gradient-exchange, providing new pathways for model synthesis and growth with divergent technical motivations and outcomes.

1. Fundamental Definitions and Distinctions

Gradient-Free Aggregation (GFA) in Federated Learning: In conventional federated learning schemes (e.g., FedAvg, FedProx), clients compute and transmit explicit gradients or parameter deltas, enabling the central server to perform model updates. GFA, in contrast, forgoes any transmission of gradients or weights. Instead, it leverages a likelihood-free Bayesian framework in which the server postulates a generative model $p(x|\theta)$, proposes candidates $\theta^*$, and updates its posterior solely through comparison of summary statistics returned by clients—derived from their private data via learned local encoders. Clients never expose gradients or model parameters, only scalar discrepancy metrics between local and synthetic data summaries (Hahn et al., 2020).

Gradient Flow Aggregation in Stochastic Growth: In geometric random growth processes, GFA refers to a deterministic rule by which new particles, introduced from random directions at infinity, are accreted to a cluster via integration along the gradient of a power-law (or logarithmic) interaction energy. Here, "aggregation" is achieved not by classic random walks or stochastic paths (as in DLA), but by deterministic gradient descent in the potential $E(x) = \sum_{i=1}^n \frac{1}{\|x-x_i\|^\alpha}$, with the location of attachment determined by the trajectory’s first contact with the existing cluster (Steinerberger, 2023).

2. Methodologies: Algorithmic and Analytical Frameworks

Federated Learning GFA via Approximate Bayesian Computation

• The server maintains a prior $p(\theta)$ on parameters of the global generative model.
• For each proposal $\theta^*$:
  • Synthetic data $X^{gen}$ is simulated using $p(\cdot|\theta^*)$.
  • Synthetic data is partitioned and dispatched to client nodes.
  • Clients encode both their private data and $X^{gen}_i$ via their local SuffiAE encoder $S_i(\cdot)$, which is a hybrid variational autoencoder/classifier trained for sufficiency and privacy.
  • The scalar discrepancy $\Delta(s_i, s^{gen}_i)$ (e.g., $\ell_2$ norm) between encoded client data and synthetic data is computed and sent to the server.
• The server aggregates discrepancies, and accepts sets of $\theta^*$ minimizing overall discrepancy, thus drawing from an approximate global posterior $p(\theta \mid \cup_i X_i)$ using an ABC kernel-smoothing scheme (Hahn et al., 2020).

Stochastic Growth GFA via Gradient Flows

• Each new particle’s trajectory is initialized at large radius in a uniformly random direction.
• The particle's position evolves according to the ODE:

$$\frac{dx}{dt} = \nabla E(x), \quad E(x) = \sum_{i=1}^n \|x-x_i\|^{-\alpha}$$

• The particle stops and is incorporated into the cluster when it reaches unit distance from the closest existing particle.
• This deterministic evolution is repeated for each subsequent particle, yielding clusters whose macroscopic properties depend critically on the parameter $\alpha$ (Steinerberger, 2023).

3. Construction of Sufficient and Private Summaries: SuffiAE

A central technical enabler for GFA in federated learning is SuffiAE, a locally trained neural encoder/decoder with the following properties:

• Input Mapping: $q_\phi: X \rightarrow z \in \mathbb{R}^d$ (encoder).
• Decoder: $f_\theta: z \rightarrow \hat{X}$, minimizes reconstruction loss.
• Classifier: $g_\psi: z \rightarrow \hat{y}$, with cross-entropy penalty.
• Training Objective:

$$\mathcal{L}(X, y; \theta, \phi, \psi) = \mathbb{E}_{z \sim q_\phi(z|X)}[\log p(X|f_\theta(z))] + \mathbb{E}_{z \sim q_\phi(z|X)}[\log p(y|g_\psi(z))] - KL[q_\phi(z|X) \| p(z)]$$

• Gaussian prior on $z$, per-instance noise $\eta \sim \mathcal{N}(0, \alpha I)$ for robustness.
• By maximizing the variational bound, $z$ retains all predictive information for $y$ (and thus for $\theta$ under standard modeling assumptions). This makes $z$ sufficient for ABC-based inference; i.e., $p(\theta \mid z) = p(\theta \mid X)$. Privacy is enforced because only this compressed summary $z$ is ever transmitted, with the decoder never leaving the client device (Hahn et al., 2020).

4. Empirical Results and Benchmarking

Federated Learning GFA (GRAFFL):

Dataset/Setting	Baseline (Raw)	After GRAFFL Augmentation	Global Oracle Upper Bound
Synthetic tri-modal (n=9000)	---	GRAFFL modes recover true parameters	---
PhysioNet2012 (AUC, 3 clients)	0.55–0.67	0.78–0.82	0.8158
Vehicle (F1, low-data sites)	0	≈1.0	---

Local AUC and F1 scores improve to near-oracle performance after GRAFFL-based augmentation, even in highly imbalanced or data-scarce regimes, without exchanging raw data or gradients (Hahn et al., 2020).

Stochastic Growth GFA:

• For $\alpha=0$, resulting clusters are round, with growth rate $\mathrm{diam}\sim n^{1/2}$.
• For general $0 \leq \alpha \leq 1$, cluster diameter satisfies

$$\mathrm{diam}\{\vec{x}\} \leq c_\alpha n^{\frac{3\alpha+1}{2\alpha+2}}$$

with sub-ballistic scaling; for $\alpha\gg 1$, clusters become sparse arms ("spidery" trees) and may exhibit ballistic growth $\mathrm{diam}\sim n$ (Steinerberger, 2023).

5. Privacy Analysis and Security Properties

• No gradients or weights are communicated in federated GFA; only low-dimensional discrepancies are exposed.
• SuffiAE encoders ensure that summary statistics provide no tractable inversion path to recover private data; dense-to-sparse mapping ($d \ll D$) and application of local noise further harden this property.
• The attack surface for gradient-inversion and membership inference is thus eliminated or drastically reduced relative to standard FL. Empirically, GFA leaks orders of magnitude less information than gradient-based protocols in comparable white-box settings. While no formal $(\epsilon,\delta)$-differential privacy is guaranteed, observed empirical leakage is negligible (Hahn et al., 2020).

6. Geometric and Theoretical Insights in Growth Models

Tree Shape and Growth Exponents:

• $\alpha=0$ yields maximally "full," disk-like trees, new particles have high probability of attaching anywhere on the boundary.
• $\alpha$ slightly positive introduces bias; longer tips accrue more particles.
• Large $\alpha$ sharply segregates growth to the extremal arms.
• Theoretical Beurling-type and Kesten-style estimates bound attachment probabilities and validate sub-ballistic growth for $0 \leq \alpha < 1$. For $\alpha\to\infty$, only convex hull particles receive accretions, with attachment probabilities determined by geometric opening angles (Steinerberger, 2023).

Higher Dimensions:

• Analogous growth and diameter exponents arise in $\mathbb{R}^d$ with $E(x) = \sum_{i=1}^n \|x-x_i\|^{-(d-2)}$:

$$\max_i \mathbb{P}\{\text{new hits }x_i\} \leq c_d\, n^{1/d-1},\quad \mathrm{diam} \leq c_d n^{(d-1)/d}$$

These match lower bounds from packing arguments (Steinerberger, 2023).

7. Significance and Implications

Gradient-Free Aggregation represents a fundamental paradigm shift in both privacy-preserving distributed learning and random cluster growth models. In learning, GFA bypasses exposure to gradient-related privacy attacks, producing competitive global models via simulation-based Bayesian inference and privately learned sufficient statistics. In geometric growth, GFA replaces highly randomized aggregation with deterministic field-driven flows, yielding tractable mathematical analysis and rich structural phenomena. The convergence of these methodologies highlights broader applications of gradient-free reasoning in both probabilistic inference and complex system modeling (Hahn et al., 2020, Steinerberger, 2023).