EdgeFD: Edge-Based Fault Diagnosis & Federated Distillation

• EdgeFD is a set of edge-centric frameworks that include a drift-aware fault diagnosis system using Beta CUSUM and DAWC, and a federated distillation method employing KMeans-based density estimation.
• The fault diagnosis component leverages real-time sensor data and Fisher-regularized continual learning to achieve high accuracy and mitigate catastrophic forgetting in IIoT applications.
• The federated distillation framework optimizes model training on resource-constrained devices by reducing communication overhead and handling strong non-IID data through local proxy filtering.

EdgeFD refers to distinct frameworks characterized by edge-computing-oriented techniques for either continual drift-aware fault diagnosis in industrial IoT or communication-efficient federated model distillation under data heterogeneity. Two unrelated systems are prominent under this designation: (1) "EdgeFD: An Edge-Friendly Drift-Aware Fault Diagnosis System for Industrial IoT" (Jiao et al., 2023) and (2) "EdgeFD: Federated Distillation on Edge Devices" (Mujtaba et al., 20 Aug 2025). Each addresses edge deployment bottlenecks—catastrophic forgetting under nonstationary conditions in the former, efficient collaborative learning under non-IID client distributions in the latter—through algorithmic and systems-level advances compatible with resource-constrained hardware.

1. Drift-Aware Fault Diagnosis System for IIoT (Jiao et al., 2023)

This EdgeFD implements real-time industrial fault diagnosis under data drift on edge devices. The architecture integrates a lightweight drift detection module based on classifier confidence statistics with a continual learning module leveraging drift-aware weight consolidation (DAWC), facilitating adaptation to evolving operational conditions.

Architecture

• Input: Sensor vibration signals (bearing health monitoring).
• Backbone: WDCNN (wide and deep convolutional neural network).
• Inference cycle: For each sample $x_i$, the system predicts the class label $\hat{y}_i$ and the confidence $q_i = \max_k p(y=k|x_i)$.
• Modules:
  • Drift Detection: Maintains a sliding window of $N$ recent confidences and inputs them to a two-distribution CUSUM scheme (Beta-based).
  • Continual Learning (DAWC): When drift is detected, selectively fine-tunes the model on a small buffer while preserving previously learned information via a Fisher Information Matrix–regularized objective.

Drift Detection Algorithm

EdgeFD’s drift detection leverages the likelihood-ratio between Beta distributions fit to older and newer confidences in the confidence window.

For window $Q = \{q_1, ..., q_N\}$:

• For $k=\delta$ to $N-\delta$, split $Q$ into reference $Q_r = \{q_1, ..., q_k\}$ and target $Q_t = \{q_{k+1}, ..., q_N\}$.
• Fit Beta distributions via MLE: $\hat{y}_i$0 and $\hat{y}_i$1.
• Compute $\hat{y}_i$2.
• Trigger drift if maximum $\hat{y}_i$3 over allowed $\hat{y}_i$4 exceeds threshold $\hat{y}_i$5.

The approach has low computational and memory demands (storage of $\hat{y}_i$6 scalars, no full-data history required).

Continual Learning via Drift-Aware Weight Consolidation (DAWC)

DAWC adapts Elastic Weight Consolidation (EWC):

$\hat{y}_i$7

• $\hat{y}_i$8: Cross-entropy on buffer accumulated post-drift.
• $\hat{y}_i$9: Fisher information for parameter $q_i = \max_k p(y=k|x_i)$0 from earlier task $q_i = \max_k p(y=k|x_i)$1, quantifying parameter importance.
• $q_i = \max_k p(y=k|x_i)$2: Optimal parameter on previous buffer/task.
• Only diagonal FIM is stored per step, avoiding high memory cost.

The process enables fine-tuned adaptation while regularizing parameter updates according to their importance for previous regimes, mitigating catastrophic forgetting.

Edge-Optimized Implementation

• Model size: ~1–2 MB.
• Typical inference time: ≤10 ms on mid-range edge devices.
• Fine-tuning only on detected drifts (few per deployment), with 10–50 gradient steps, and no storage of previous raw data.
• Compared to standard TL, achieves 20–30% reduction in fine-tuning compute.

Experimental Evaluation

On the CWRU bearing fault dataset (10 classes, representing combinations of fault types, sizes, and loads):

Method	AA (%)	AF
STL	81.25 (±0.63)	0.22 (±0.02)
FCB (TL)	86.67 (±0.72)	0.16 (±0.02)
MAML	89.28 (±0.16)	0.14 (±0.01)
DAWC (ours)	93.92 (±0.15)	0.07 (±0.01)

AA: Average Accuracy; AF: Average Forgetting.

DAWC achieves the highest accuracy and lowest forgetting through all environmental drifts. Drift points are detected immediately; adaptation incurs ~20–40% less compute cost than baselines.

Visualization and Diagnosis Platform

• Web-based UI: Streams real-time class confidences and drift signals.
• Upon drift, special indicator pauses inference and triggers adaptation.
• Alerts (SMS/email) on confirmed faults.
• Real-time plots of vibration and drift histories.

2. Federated Distillation on Edge Devices for Non-IID Data (Mujtaba et al., 20 Aug 2025)

The second EdgeFD addresses federated model training under strong non-IID conditions. This framework introduces an efficient, KMeans-based local proxy-selection/density estimation for collaborative knowledge distillation, fully eliminating server-side ambiguity filtering.

Federated Distillation Workflow

• Clients: Edge devices with local, potentially heterogeneous models.
• Proxy set exchange: Each client shares a small, random subset of private data (proxy set) with the server, aggregated and redistributed for cross-client distillation.
• Rounds:
  • Each client trains a KMeans model on its features to summarize the in-distribution (ID) data manifold.
  • For each proxy batch, client computes the minimum distance from sample to centroids: if below threshold $q_i = \max_k p(y=k|x_i)$3, marks the proxy as ID; otherwise, out-of-distribution (OOD).
  • Only soft logits from ID proxies are communicated to the server.
  • The server averages ID logits across clients and redistributes them.
  • Clients then update models with local data loss and Kullback–Leibler distillation loss using aggregated peer logits.

KMeans-Based Density-Ratio Estimation (KMeans-DRE)

• Computational cost: Linear in local sample size and feature dimension.
• Classification: $q_i = \max_k p(y=k|x_i)$4 labeled as ID if $q_i = \max_k p(y=k|x_i)$5.
• Compared to kernel-based DRE (e.g., KuLSIF), KMeans-DRE requires significantly lower compute and memory ($q_i = \max_k p(y=k|x_i)$6 versus $q_i = \max_k p(y=k|x_i)$7 or worse).

A plausible implication is that KMeans-DRE is practical for edge hardware, including CPUs <$q_i = \max_k p(y=k|x_i)$8 GHz and low memory footprints.

Experimental Highlights

• 10 clients; datasets: MNIST, FashionMNIST, CIFAR-10.
• Heterogeneous client models; scenarios include:
  • Strong non-IID (each client: single label)
  • Weak non-IID (random 3-label subsets)
  • IID (random split)
• Baselines: FedMD, FedED, DS-FL, FKD, PLS, Selective-FD (using kernel DRE)
• EdgeFD results (test accuracy after 200 rounds):

Scenario	MNIST	FashionMNIST	CIFAR-10
Strong Non-IID	98.92%	88.55%	82.57%
Weak Non-IID	98.88%	88.74%	84.88%
IID	99.08%	89.90%	86.37%

EdgeFD’s accuracy approaches that of IID baselines, even with strong label skew. Server-side filtering is eliminated, reducing protocol latency by 15–25% compared to two-stage approaches.

Communication and Practical Considerations

• Logit communication per round is 10–20% of full model gradient size.
• Client-centric filtering avoids excess latency and server overhead.
• EdgeFD supports model heterogeneity and personalized training.
• Limitations include possible data leakage from proxy sharing and the need for feature extractor pretraining on high-dimensional data (e.g., ResNet-18 on CIFAR-10).
• Selection of threshold $q_i = \max_k p(y=k|x_i)$9 is critical for trade-off between OOD contamination and loss of ID samples; auto-tuning remains unresolved.

3. Key Algorithmic Principles

Component	Drift-Aware FD (IIoT) (Jiao et al., 2023)	Federated Distillation (Non-IID) (Mujtaba et al., 20 Aug 2025)
Data drift handling	Beta CUSUM on confidences + DAWC	Data heterogeneity addressed via KMeans-DRE filtering
Continual learning	EWC-style weight consolidation	Deep distillation loss across ID proxies
Edge efficiency	Minimal state; adaptation on drift only	On-device clustering; no server filtering
Avoiding forgetting	Fisher-matrix-regularized fine-tuning	Exclusion of OOD proxies

Context

The DAWC method in (Jiao et al., 2023) shows an application of regularized continual learning to maintain high accuracy under nonstationary industrial conditions, while the KMeans-DRE approach in (Mujtaba et al., 20 Aug 2025) demonstrates a practical framework for robust model distillation under severe non-IID data splits, both targeting edge deployment constraints.

4. Applications and Impact

• IIoT Fault Diagnosis: Real-time, reliable monitoring of industrial bearings; enables maintenance scheduling and rapid identification of system degradation.
• Federated Edge Learning: Collaborative training of models on distributed edge devices with heterogeneous, private datasets; mitigates negative transfer from data heterogeneity.
• Both EdgeFD systems are designed for minimal memory, compute, and communication overhead, enabling deployment in environments with strict resource budgets.

5. Limitations and Open Challenges

Fault Diagnosis (IIoT)

• The approach does not require replay buffers but depends on the accuracy of confidence-based drift detection, which could be sensitive to parameter and buffer window choices.

Federated Distillation

• Potential leakage from proxy data sharing; countermeasures such as differential privacy are not implemented.
• KMeans-DRE thresholding is sensitive; auto-tuning remains an open problem.
• Requires pretrained feature extractors for unstructured, high-dimensional data (e.g. images).
• Assumes uniform proxy contribution from clients; adaptation to real-world client heterogeneity may require further mechanisms.

6. Conclusion

EdgeFD denotes two frameworks designed for edge-centric analytics under nonstationary or non-IID settings. The drift-aware IIoT system employs confidence-based detection and Fisher-regularized continual learning to sustain accuracy under shifting conditions and computational constraints (Jiao et al., 2023). The federated distillation variant utilizes a computationally efficient KMeans-DRE to enable robust collaborative knowledge transfer on heterogeneous, resource-constrained clients, achieving near-IID performance under severe label skew and communication limitations (Mujtaba et al., 20 Aug 2025). Both exemplify the integration of algorithmic innovation with edge deployment realities.