# FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions

Source: [FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions](<https://www.sota2.com/research/paper/68fa6b7c1e209131cea2e7c3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:49:06.844Z

Authors: Behnam Ghavami, Mani Sadati, Zhenman Fang, Lesley Shannon

Published: 2021-12-27T07:07:50.000Z

## Abstract

Deep neural networks (DNNs) are increasingly being deployed in safety-critical systems such as personal healthcare devices and self-driving cars. In such DNN-based systems, error resilience is a top priority since faults in DNN inference could lead to mispredictions and safety hazards. For latency-critical DNN inference on resource-constrained edge devices, it is nontrivial to apply conventional redundancy-based fault tolerance techniques. In this paper, we propose FitAct, a low-cost approach to enhance the error resilience of DNNs by deploying fine-grained post-trainable activation functions. The main idea is to precisely bound the activation value of each individual neuron via neuron-wise bounded activation functions so that it could prevent fault propagation in the network. To avoid complex DNN model re-training, we propose to decouple the accuracy training and resilience training and develop a lightweight post-training phase to learn these activation functions with precise bound values. Experimental results on widely used DNN models such as AlexNet, VGG16, and ResNet50 demonstrate that FitAct outperforms state-of-the-art studies such as Clip-Act and Ranger in enhancing the DNN error resilience for a wide range of fault rates while adding manageable runtime and memory space overheads.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-10 (test) (Accuracy Drop)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-test-accuracy-drop.md>): Image Classification; CIFAR-10 (test); Accuracy Drop: 1.53.

- [Image Classification on CIFAR-100 (test) (Accuracy Drop)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-test-accuracy-drop.md>): Image Classification; CIFAR-100 (test); Accuracy Drop: 6.31.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2112.13544>)

[PDF](<https://arxiv.org/pdf/2112.13544.pdf>)