- The paper introduces a dual-timescale framework that combines online layer-selective rehearsal for training-time forgetting with meta-trained, single-pass head recovery for long-term federated drift.
- Layer sensitivity analysis shows that the segmentation head causes 87.1% of measured forgetting, while the method improves cumulative mIoU by up to 8.3 points over the strongest federated baseline across three Mars datasets.
- The framework recovers degraded models in 5–8 epochs versus 11–18 for retraining and delivers 2.0–3.3% higher cumulative mIoU than a competing method on a four-rover physical testbed.
Motivation and problem setting
This paper addresses federated continual learning (FCL) for mobile autonomous fleets, using Mars rover terrain segmentation as the driving application. The setting is a fleet of K rovers that must incrementally learn new terrain classes as they traverse geologically distinct regions, under strict Non-IID data distributions and communication budgets that preclude centralized training on Earth. The authors formulate a federated class-incremental semantic segmentation problem in which label spaces across tasks are strictly non-overlapping, exemplar memory buffers are fixed-capacity, and performance is measured by cumulative mIoU over all classes after the final task.
The paper identifies two distinct degradation timescales: training-time forgetting, arising from gradient conflicts between new-task data and memory rehearsal during local updates, and long-term cumulative drift, induced by repeated FedAvg-style aggregation of Non-IID client models over the mission lifecycle. The central claim is that prior FCL methods fail on three counts: they apply uniform layer-wise protection despite heterogeneous forgetting sensitivities; their recovery mechanisms (LoRA, distillation, variational inference) require iterative optimization with high overhead; and their evaluation relies on simulations that ignore real system heterogeneity.
Empirical motivation: layer-wise forgetting heterogeneity
A controlled layer-replacement experiment on MarsScapes under extreme Non-IID conditions (β=0.1, K=30) provides the empirical foundation for the design. Replacing each layer of a Task-0 model individually with its Task-t counterpart reveals sharply asymmetric contributions to forgetting:
| Layer group |
Share of forgetting magnitude |
Effect on Task-0 mIoU |
| Segmentation head |
87.1% |
45.0% → 21.4% |
| Shallow layers |
6.5% |
−1.4 pp |
| Deep backbone layers |
−6.9% |
45.0% → 46.5% |
Two findings are notable. First, the segmentation head dominates forgetting by more than an order of magnitude, which directly motivates head-centric stabilization and recovery. Second, deep-layer replacement improves old-task performance, implying that uniform rehearsal constraints are actively harmful for the backbone — strong constraints suppress beneficial adaptation while weak constraints leave the head unprotected. This is the strongest empirical claim in the paper and the basis for stratified rather than uniform protection.
Proposed framework
The framework couples two components operating at the two timescales.
Layer-Selective Rehearsal (LSR) operates during local training via three learned generators ϕs, ϕd, ϕc, one per layer group. Each generator consumes compact per-channel statistics of parameters and gradients plus memory-derived features hm (and, for the head generator, class prototypes and inter-class angular features), producing per-channel correction terms applied as θl←θl−η∇l+αlΔθl with stratified weights αs<αd<αc. Unlike prior binary freeze/adapt masks or one-shot post-hoc sensitivity scoring, the corrections are produced continuously by generators trained online alongside model parameters, so per-layer adjustment adapts as training proceeds. Generator parameters remain client-local and never enter the communication budget; the full auxiliary footprint is 2.57 M parameters (10.2 MB), about 9.6% of the baseline DeepLabV3+ model.
Rapid Knowledge Recovery (RKR) addresses long-term drift with a single-pass recovery function β=0.10 acting exclusively on the segmentation head. It fuses encoded degraded head parameters, class prototypes, memory exemplars, and geometric features through attention-weighted aggregation to produce a direct correction β=0.11. Crucially, β=0.12 is meta-trained once during Task 0 via episodic procedures that simulate degradation by partitioning the Task-0 label space into training and held-out subsets, then optimizing a bi-level objective on recovery quality. The learned patterns transfer zero-shot to all subsequent tasks without retraining or additional communication; deployment triggers when a rover's cumulative mIoU falls below a threshold set at 80% of Task-0 mIoU, followed by brief fine-tuning on the local buffer.
Theoretical analysis
The theory section is explicitly framed as qualitative design guidance rather than tight bounds, an honest positioning given idealized assumptions (layer-wise smoothness, bounded gradient variance). Three results carry the argument:
- Layer-wise forgetting decomposition: a Taylor-based decomposition expresses task-level forgetting as a weighted sum of per-layer replacement effects with residual β=0.13, providing the structural counterpart to the replacement experiment.
- Suboptimality of uniform rehearsal: under matched average regularization, stratified rehearsal achieves cumulative forgetting bounded by β=0.14 times that of any uniform-β=0.15 strategy, yielding a constant-factor improvement when head dominance holds.
- Inevitable long-term degradation: even assuming a perfect training-time defense (β=0.16), aggregation-induced drift under Non-IID heterogeneity combined with diminishing memory coverage produces cumulative degradation lower-bounded by β=0.17 for fixed buffer size. This result justifies the necessity of a recovery component: prevention alone cannot bound long-horizon degradation.
A sample-complexity corollary compares RKR against retraining: recovery requires β=0.18 fine-tuning epochs versus β=0.19 for full retraining, though the authors concede the logarithmic bound relies on a local strong-convexity approximation that does not strictly hold for deep networks.
Simulation results
Experiments use three Mars datasets — MarsScapes (18,460 panoramic images, 9 classes), S5Mars (6,000 high-resolution images including navigation-critical hole/ridge classes), and AI4MARS (~35,000 images spanning three landing sites) — under 5-1, 3-2, and 2-1 incremental splits with K=300 clients. All baselines share identical FedAvg aggregation and communication budgets, with method-specific mechanisms kept client-local.
Headline results:
| Dataset / setting |
Ours |
Best baseline (FBL) |
Fine-tuning |
| MarsScapes 5-1 |
30.4% |
27.6% |
6.1% |
| MarsScapes 3-2 |
27.8% |
24.4% |
8.9% |
| S5Mars 5-1 |
42.2% |
40.8% |
10.5% |
| S5Mars 3-2 |
39.2% |
30.9% |
9.4% |
| AI4MARS 2-1 |
33.5% |
29.6% |
9.3% |
The largest margin over the strongest federated baseline is 8.3 points (S5Mars 3-2), and up to 31.7 points over fine-tuning. Under extreme Non-IID (K=301), several baselines collapse to zero IoU on incremental classes because newly learned classes are averaged out during aggregation, while the proposed method retains them. Ablations confirm the stratification claim: removing the head generator costs 9.2–11.0 mIoU, whereas removing the shallow generator costs only 0.4–0.5, mirroring the sensitivity analysis. Uniform rehearsal loses 3.4–4.9 points relative to full LSR, with the gap widening under severe Non-IID.
Recovery efficiency comparisons show RKR restoring degraded models in 5–8 epochs (e.g., MarsScapes from 14.4% to 41.4% in 6 epochs) versus 11–18 epochs for full retraining and 15–20 for variational distillation — a 2–3× speedup attributable to zero-shot transfer of Task-0-learned recovery patterns. The trigger threshold is not hand-tuned per dataset; sweeping it over 70–90% of Task-0 mIoU changes final mIoU by less than 0.8%, with no spurious or missed activations observed.
Real-world testbed validation
A physical testbed comprising four MentorPi rovers (Raspberry Pi 5, Cortex-A76) and an edge server running Flower validates the full lifecycle under realistic constraints. The authors are explicit about scope: the Raspberry Pi proxies radiation-hardened flight CPUs, WiFi 802.11ac proxies UHF relay links without contact-window structure, and the four-client scale does not verify behavior at K=302. Within this scope, the framework achieves 2.0–3.3% higher cumulative mIoU than FBL paired with meta-learning distillation across complete task sequences, maintains 0.8–2.5% advantages at K=303, and retains 0.9–5.8% advantages under high memory-buffer heterogeneity (K=304).
The paper also concedes that real missions may present domain-incremental and class-overlap conditions beyond the non-overlapping class-incremental setting evaluated here; the framework is argued to be structurally compatible with both, but no public Mars benchmark currently exists to quantify such extensions.
Limitations and open questions
Several limitations bear directly on the strength of the claims. The theoretical guarantees rest on smoothness and bounded-variance assumptions acknowledged as idealized for deep segmentation networks under strong heterogeneity, and the logarithmic recovery bound depends on a local strong-convexity approximation. The testbed's four-client scale and proxy hardware mean system-level conclusions at fleet scale rest on simulation. The evaluation is confined to non-overlapping class-incremental splits; domain-incremental robustness remains unquantified. Open questions include whether the learned generators scale gracefully to substantially larger class counts than the nine-class benchmarks tested, and whether the Task-0 meta-trained recovery function degrades when later-task degradation patterns diverge substantially from the episodic simulations used in meta-training.
Conclusion
This paper contributes a dual-timescale FCL framework that coordinates stratified, online-learned rehearsal against training-time forgetting with single-pass, meta-learned recovery against long-term aggregation-induced drift. Its principal strengths are the quantified demonstration of extreme layer-wise forgetting asymmetry (87.1% concentrated in the segmentation head), a formal argument that prevention alone cannot bound long-term degradation, consistent gains of up to 8.3 mIoU over the strongest federated baseline across three Mars benchmarks, and rare physical-testbed validation under Non-IID and resource heterogeneity. The work establishes lifecycle-aware coordination of prevention and recovery as a necessary design principle for FCL in extended autonomous missions, while leaving domain-incremental generalization and large-fleet behavior as open empirical questions.