DroneFL: Federated Multi-UAV Tracking

• DroneFL is a federated learning framework for multi-UAV visual target tracking that combines a frozen YOLO backbone with a lightweight transformer for efficient trajectory prediction.
• It addresses onboard computational limits and data heterogeneity by employing altitude-conditioned adaptive normalization and position-invariant design.
• Empirical evaluations show significant reductions in prediction error (up to 96%) and tracking distance, demonstrating real-time performance and improved generalization compared to traditional methods.

Searching arXiv for the specified papers and closely related UAV federated learning work. arxiv_search(query="(Yu et al., 25 Sep 2025) DroneFL Federated Learning for Multi-UAV Visual Target Tracking", max_results=5, sort_by="relevance") arxiv_search(query="DroneFL Federated Learning for Multi-UAV Visual Target Tracking", max_results=10) DroneFL is a federated learning framework for efficient multi-UAV visual target tracking, introduced for coordinated monitoring of dynamic entities in applications such as precision agriculture, environmental monitoring, disaster response, and security surveillance. It addresses three coupled difficulties that are explicit in the formulation: limited onboard computational resources, significant data heterogeneity caused by varying targets and fields of view, and the need to couple trajectory prediction with multi-robot planning. The framework combines a lightweight local predictor, periodic cloud aggregation, heterogeneity-aware normalization, cloud-side prediction fusion, and perception-aware trajectory optimization, and is described as the first federated learning framework specifically designed for efficient multi-UAV target tracking (Yu et al., 25 Sep 2025).

1. Federated objective and tracking formulation

DroneFL follows the standard FedAvg recipe, adapted to $M$ UAVs collaboratively training a shared trajectory-prediction model $\phi$. The trainable global parameter vector is $w\in\mathbb{R}^d$, corresponding to the shallow transformer component. For UAV $k\in\{1,\dots,M\}$, the local empirical loss is

$$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$

where $\mathcal T_k$ denotes observation timestamps, $(x^k_{t_p},\gamma^k_{t_p})$ are past odometry and image inputs, and $z^k_{t_f}$ are the (pseudo-)ground-truth future positions. The global objective is a weighted sum of local losses,

$$F(w)\;=\;\sum_{k=1}^M p_k\,F_k(w), \quad p_k=\frac{|\mathcal T_k|}{\sum_{i=1}^M|\mathcal T_i|}.$$

The training protocol is organized into global rounds. In round $r$, the server broadcasts the current model $\phi$0 to every UAV; each UAV performs $\phi$1 epochs of local SGD beginning from $\phi$2,

$\phi$3

and the server then aggregates updates through FedAvg,

$\phi$4

This formulation is notable because the learned predictor is not an isolated forecasting module. Its output is consumed by the downstream multi-UAV control layer, so the federated objective is embedded in an end-to-end tracking system rather than treated as a standalone offline learning task (Yu et al., 25 Sep 2025).

2. Lightweight local predictor under onboard constraints

To satisfy onboard resource limits, DroneFL uses a two-stage local predictor. A frozen YOLOv11n backbone processes the RGB image $\phi$5 and produces a set of bounding boxes $\phi$6. A lightweight transformer $\phi$7, consisting of two encoders, two decoders, and hidden dimension $\phi$8, takes the recent history of bounding boxes and odometry $\phi$9 and outputs a short $w\in\mathbb{R}^d$0-step trajectory $w\in\mathbb{R}^d$1. During federated learning, only $w\in\mathbb{R}^d$2 is updated; the YOLO weights remain fixed (Yu et al., 25 Sep 2025).

This separation of perception and trajectory modeling serves two purposes that are explicit in the reported design. First, it reduces the trainable parameter set to the shallow transformer, which lowers communication load during FL because only transformer weights are exchanged. Second, it preserves a vision front end whose inference cost is predictable on lightweight hardware. The architecture therefore implements a resource-aware division between a fixed detector and a trainable temporal forecaster.

Ablation results reported for the framework indicate that the frozen-backbone plus shallow transformer design is essential for fast convergence and for end-to-end gains in prediction and control. This suggests that DroneFL is not merely a generic FedAvg deployment over a vision stack; the local model topology is part of the method’s core contribution rather than an incidental engineering choice (Yu et al., 25 Sep 2025).

3. Position invariance, altitude conditioning, and non-IID mitigation

The principal non-IID factor identified for vision-based FL in multi-UAV tracking is variation in camera viewpoints, especially altitude. DroneFL addresses this with a position-invariant model architecture and altitude-conditioned adaptive instance normalization (AdaIN). Within each transformer block, if $w\in\mathbb{R}^d$3 is an intermediate feature map and the drone altitude is $w\in\mathbb{R}^d$4, style parameters are computed as

$w\in\mathbb{R}^d$5

and each channel $w\in\mathbb{R}^d$6 is normalized by

$w\in\mathbb{R}^d$7

By shifting and scaling feature statistics based on altitude, the model is described as becoming invariant to field-of-view changes caused by different flight heights. DroneFL also recenters all $w\in\mathbb{R}^d$8 coordinates relative to the UAV’s current position, yielding the position-invariant architecture (Yu et al., 25 Sep 2025).

The empirical motivation for this design is explicit. When two UAVs share targets but fly at different heights, the local MSE loss on one agent fluctuates sharply after each aggregation step, indicating severe heterogeneity. The proposed invariance mechanisms reduce the gap between local and global objectives. In three increasingly challenging simulation cases—2 UAVs and 2 targets with constant speed, 3 UAVs and 3 targets with random walk, and 4 UAVs and 6 targets—DroneFL cuts the final test MSE by 47%–96% compared to FedAvg, FedProx, and FedPer, and approaches the centralized upper bound. The reported interpretation is therefore architectural: heterogeneity is not treated only by altering the optimizer, but by encoding viewpoint structure directly into the predictor (Yu et al., 25 Sep 2025).

A common misconception in federated learning for robotics is that non-IID behavior can be handled adequately by aggregation rules alone. The DroneFL results point in a different direction. A plausible implication is that, for embodied visual systems, invariance at the representation level can be as important as the federation algorithm itself, particularly when the dominant heterogeneity arises from geometry rather than solely from label or class imbalance.

4. Cloud-side fusion and perception-aware multi-UAV planning

At inference time, each UAV $w\in\mathbb{R}^d$9 sends its transformer’s $k\in\{1,\dots,M\}$0-step trajectory prediction $k\in\{1,\dots,M\}$1 to the cloud. The server maintains an adaptive-innovation Extended Kalman Filter per target $k\in\{1,\dots,M\}$2. For UAV $k\in\{1,\dots,M\}$3, the innovation is

$k\in\{1,\dots,M\}$4

The measurement noise $k\in\{1,\dots,M\}$5 is updated by an exponential moving average of $k\in\{1,\dots,M\}$6, gated by Mahalanobis distance to reject outliers, and the fused estimate $k\in\{1,\dots,M\}$7 together with covariance $k\in\{1,\dots,M\}$8 follows the standard EKF corrections (Yu et al., 25 Sep 2025).

The fused forecasts $k\in\{1,\dots,M\}$9 are then passed to a joint control optimizer. The server solves for the control sequence $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$0 by minimizing

$$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$1

subject to the linear motion model $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$2 and feasible sets $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$3. The perception-aware cost is

$$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$4

which penalizes distance to the target while enforcing a minimum safe standoff $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$5, and also penalizes predicted uncertainty through $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$6.

This coupling of learned prediction, Bayesian fusion, and constrained control is central to DroneFL’s scope. It is not only a federated predictor, but a tracking system in which the cloud layer shares two roles: global model aggregation during training and multi-agent estimation-and-planning during inference. The planning objective makes explicit that target proximity and forecast uncertainty are co-optimized rather than handled as separate modules (Yu et al., 25 Sep 2025).

5. Real-time implementation and reported empirical performance

DroneFL is reported to run end-to-end on a Raspberry Pi 5 under realistic communication constraints. Inference latency per frame is 0.13 s for the YOLO backbone and 0.01 s for the transformer, corresponding to 5 Hz real-time tracking. Local training requires 1.76 s per epoch on device using Adam, with 10 epochs per FL round. The energy cost is 0.78 J per inference for YOLO, 0.11 J per inference for the transformer, and 14.11 J per training epoch. Communication consists of transformer weights every 100 s together with $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$7-step position vectors in real time, averaging 1.56 KB/s; the centralized alternative of streaming raw images requires 2.46 MB/s (Yu et al., 25 Sep 2025).

The experiments are conducted in a simulated agricultural testbed with cases of rising difficulty. Relative to a fixed-distributed baseline without federated learning, prediction MSE is reduced by 6%–83%; one example in Case 2 reports a reduction from $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$8 to $$F_k(w)\;=\;\frac{1}{|\mathcal T_k|}\sum_{t\in\mathcal T_k}\!\bigl\|\phi_w(x_{t_p}^k,\gamma_{t_p}^k)-z_{t_f}^k\bigr\|^2,$$9 m$\mathcal T_k$0. Average tracking distance falls by 0.4%–4.6%; one example in Case 3 reports a reduction from $\mathcal T_k$1 to $\mathcal T_k$2 m$\mathcal T_k$3. Compared with centralized training, DroneFL is reported to generalize far better to a new orchard environment: centralized end-to-end models lose track entirely in complex scenarios, with MSE $\mathcal T_k$4 m$\mathcal T_k$5, whereas DroneFL remains stable. The abstract reports the same performance summary in compressed form, namely that DroneFL reduces prediction error by 6%–83% and tracking distance by 0.4%–4.6% compared to a distributed non-FL framework (Yu et al., 25 Sep 2025).

These results are significant because they combine system-level and learning-level measurements. The framework is evaluated not only through loss reduction but also through tracking distance, latency, energy, and bandwidth. This suggests that the claimed contribution is explicitly cross-layer: model design, federation strategy, and control performance are all part of the reported outcome.

6. Position within UAV federated learning and nomenclature

DroneFL (Yu et al., 25 Sep 2025) belongs to a broader literature on federated learning for UAV systems, but its architectural commitments are specific. Its aggregation is cloud-based, its task is multi-UAV visual target tracking, and its main technical mechanisms are a frozen YOLOv11n backbone, a shallow transformer, altitude-based AdaIN, EKF fusion, and perception-aware planning. By contrast, Qu et al. propose DFL-UN, a decentralized federated learning architecture for UAV networks with no always-on fusion server, where each UAV plays both worker and aggregator over an undirected A2A communication graph $\mathcal T_k$6 and performs gossip–gradient descent using a doubly-stochastic mixing matrix (Qu et al., 2021).

In DFL-UN, UAV $\mathcal T_k$7 stores local data $\mathcal T_k$8, maintains $\mathcal T_k$9, exchanges parameters with one-hop neighbors $(x^k_{t_p},\gamma^k_{t_p})$0, forms a consensus iterate, computes a local gradient, and updates its model without centralized aggregation. Under the stated assumptions that each $(x^k_{t_p},\gamma^k_{t_p})$1 is $(x^k_{t_p},\gamma^k_{t_p})$2-smooth and $(x^k_{t_p},\gamma^k_{t_p})$3-strongly convex and that the mixing matrix satisfies $(x^k_{t_p},\gamma^k_{t_p})$4, the summary gives a convergence sketch of the form

$(x^k_{t_p},\gamma^k_{t_p})$5

with a linear convergence rate for $(x^k_{t_p},\gamma^k_{t_p})$6, and $(x^k_{t_p},\gamma^k_{t_p})$7 in the nonconvex case. Its simulation study reports that the decentralized FL average loss curve lies very close to centralized FedAvg, with final gap approximately 0.0156, individual UAV loss differences at most 0.0229 after 60 rounds, and lower total latency with approximately 100 ms saving over 60 rounds because a global model need not be broadcast each round (Qu et al., 2021).

A potential source of confusion is terminological. The detailed summary of DFL-UN refers to the decentralized method as “DroneFL,” whereas the arXiv paper titled “DroneFL: Federated Learning for Multi-UAV Visual Target Tracking” designates a distinct 2025 framework centered on cloud aggregation and visual tracking (Yu et al., 25 Sep 2025). The distinction is substantive rather than cosmetic: one addresses peer-to-peer federated learning for UAV networks in general, and the other addresses heterogeneity-resilient visual trajectory prediction tightly integrated with multi-UAV planning.

The comparison also clarifies open research directions already identified in UAV FL. DFL-UN highlights convergence monitoring without a reliable central server, robustness to UAV mobility and heterogeneity, energy efficiency under frequent A2A exchanges, convergence under time-varying graphs, secure aggregation, and joint optimization of UAV trajectories, communication scheduling, and FL hyperparameters (Qu et al., 2021). DroneFL’s emphasis on cloud fusion and perception-aware control suggests a complementary design point in the same landscape: centralization can be retained when fusion quality, planning coordination, and bandwidth-efficient model exchange are prioritized, whereas decentralization targets single-point-of-failure concerns and unreliable links.