ApproxABFT: Approximate Algorithm-Based Fault Tolerance for Neural Network Processing

Abstract: With the increasing deployment of deep neural networks (DNNs) in terrestrial and aerospace safety-critical applications, system reliability has emerged as a co-equal design metric alongside computational efficiency. Algorithm-based fault tolerance (ABFT) mechanisms, characterized by architecture-agnostic and cost-effectiveness, have become a promising solution for reliability enhancement. However, conventional ABFT approaches rely on rigorous verification mechanisms where even minor computational deviations trigger error recovery processes, which not only disregards the intrinsic fault tolerance characteristics of DNN models but also incurs redundant fault tolerance processing overhead. To address these limitations, we propose an Approximate ABFT framework (ApproxABFT) that innovatively introduces adaptive error tolerance thresholds to enable selective fault recovery, activating error correction modules exclusively when computational deviations exceed predefined thresholds. This approach effectively mitigating overreaction to non-critical computational errors. Furthermore, a dynamic block granularity optimization algorithm is implemented to achieve inter-layer error sensitivity balancing. Experimental evaluations demonstrate that the proposed ApproxABFT achieves a 43.39% average reduction in redundant computing overhead compared to previous accurate ABFT, while simultaneously enhancing the tolerable soft error rate by an order of magnitude.