Data-Agnostic Robust Training (DART)

• DART is a framework that enhances model robustness by employing data-agnostic training techniques without relying on specific corruption priors.
• It utilizes methods like augmentation-consistency, optimized Gaussian noise injection, and hybrid adversarial training to mitigate diverse data and label corruptions.
• In federated learning, DART leverages server-side proxy data and a teacher-student setup to improve resilience while preserving client privacy and reducing computational load.

Searching arXiv for the cited DART-related papers and closely related context. Data-Agnostic Robust Training (DART) denotes training procedures that deliver robustness to data corruptions without relying on prior knowledge of the corruption type, mechanism, or parameters, and the cited literature uses the acronym in multiple technical settings. In federated learning, DART is a server-side, label-free robustification method that improves corruption robustness of the global federated model without accessing client data (Bekdache et al., 24 Aug 2025). In off-policy imitation learning for robotic bed-making, DART is an algorithm for learning robust policies by injecting small, optimized Gaussian noise into the supervisor’s policy during demonstrations (Laskey et al., 2017). In robust supervised learning under simultaneous data and label corruptions, ERAT is presented as a practical instantiation of DART through hybrid adversarial training, class-rebalancing sample selection, and semi-supervised consistency without prior knowledge regarding noise details (Zhang et al., 2024).

1. Terminology and scope

The cited literature uses DART in three closely related but non-identical senses. In each case, the common thread is robustness under uncertainty, but the mechanism and operating regime differ.

Setting	What DART denotes	Core mechanism
Federated learning	Server-side, label-free robustification	Augmentation-consistency plus distillation on unlabeled proxy data
Learning from Demonstrations	An LfD algorithm for learning robust policies	Optimized Gaussian noise injection into supervisor actions
Dual-corruption classification	A data-agnostic robustness paradigm	Hybrid adversarial training plus class-rebalanced semi-supervision

A common misconception is that DART refers to a single algorithmic template. The cited papers instead assign the acronym to distinct formulations adapted to federated optimization, imitation learning, and adversarially corrupted supervised learning. This suggests that DART is best understood as a family resemblance centered on robustness without direct dependence on the target clean-data distribution or on detailed corruption priors, rather than as one fixed procedure (Bekdache et al., 24 Aug 2025).

2. Federated DART in FedERL

In "FedERL: Federated Efficient and Robust Learning for Common Corruptions" (Bekdache et al., 24 Aug 2025), DART is introduced as a server-side, label-free robustification method that improves corruption robustness of the global federated model without accessing client data. The motivation is specific to federated learning: standard robust training methods such as AugMix and adversarial training typically require computationally heavy augmentations on the training data, and in federated learning that burden falls on clients. The paper reports that robust training increases FLOPs by about $3\times$ and GPU memory by about $2.5\times$, resulting in approximately $3.4\times$ higher time and $3.2\times$ higher energy on edge devices such as Jetson Orin Nano. FedERL therefore exploits the asymmetry of resources in federated learning: clients are compute- and energy-constrained and must preserve privacy, while the server can afford additional training.

The target robustness notion is common-corruption robustness under noise, blur, weather, and digital artifacts. The usual robust risk is written as

$$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$

Directly optimizing this objective on clients is described as infeasible because of resource and privacy constraints. DART replaces the inner corruption expectation by a server-side augmentation distribution and replaces labels $y$ with the teacher’s soft targets, operating on a public unlabeled proxy distribution $\mathcal{D}_0$.

The method uses a fixed teacher model $t(w_t)$, initialized to the current aggregated global model $f(w_0)$, and a student model $s(w_s)$ with identical architecture, initialized at $2.5\times$0 and updated to $2.5\times$1. For an unlabeled sample $2.5\times$2, AugMix generates two augmented views $2.5\times$3 and $2.5\times$4, while the clean $2.5\times$5 is retained. The student produces predictions $2.5\times$6, $2.5\times$7, and $2.5\times$8, and the teacher produces $2.5\times$9. The consistency loss is

$3.4\times$0

with the three-distribution Jensen–Shannon divergence defined through the barycenter

$3.4\times$1

The distillation term is

$3.4\times$2

The server minimizes

$3.4\times$3

with $3.4\times$4 in the experiments, and the optimization target is

$3.4\times$5

Early stopping uses a validation split $3.4\times$6, halting when $3.4\times$7 stops improving with patience $3.4\times$8.

The paper states that DART uses no client data, no auxiliary labels, no generative models, and no batch-norm or statistic manipulation. It is purely augmentation-consistency plus distillation on unlabeled public data. The data-agnostic mechanism is operationalized through a proxy dataset $3.4\times$9 that need not match the clients’ distribution; experiments report similar performance when $3.2\times$0 is CIFAR-100 or Tiny ImageNet, with slightly better robustness for larger $3.2\times$1 (Bekdache et al., 24 Aug 2025).

3. Optimization structure, corruption model, and efficiency in federated learning

FedERL interleaves ordinary federated training with periodic server-side DART steps. The per-round procedure is: the server broadcasts global weights $3.2\times$2; each client performs $3.2\times$3 local clean-training epochs on private data $3.2\times$4, minimizing cross-entropy and returning $3.2\times$5; the server aggregates with FedAvg,

$3.2\times$6

and every $3.2\times$7 global rounds, the server runs $3.2\times$8, initializing teacher and student from the aggregated model, optimizing $3.2\times$9 on the train split with early stopping on the validation split, and returning the updated weights (Bekdache et al., 24 Aug 2025).

The concise DART subroutine is also explicit: split $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$0 into train and validation; set $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$1 and keep $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$2 fixed; for up to $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$3 epochs, minimize $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$4 with learning rate $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$5 on the train split; early stop with patience $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$6; return the best $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$7. Client training remains standard clean training with client learning rate $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$8, $$\mathcal{R}_{\mathrm{rob}}(\theta) = \mathbb{E}_{(x,y)\sim\mathcal{D}} \; \mathbb{E}_{c\sim\mathcal{C}} \big[ \ell(f_\theta(c(x)),y) \big].$$9 local epoch per round, and $y$0 global rounds in the reported setup.

The corruption model is defined through CIFAR-10-C with severity levels $y$1. Corruptions include Gaussian, shot, and impulse noise; defocus, motion, zoom, and glass blur; snow, frost, and fog; and digital effects such as brightness, contrast, and JPEG artifacts. The corruption operator is written

$y$2

and robust accuracy $y$3 is computed on $(x_{\mathrm{corr}},y)\sim\mathcal{D