Image Classification on CIFAR-10 (IID) (Backdoor Attack Evaluation)
0.9524Average BANo Defense
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No DefenseAttack scenario=Neurotoxin, Backbone=ResNet-182025.05 | 0.9524 | 0.8682 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.9435 | 0.8823 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseAttack scenario=IBA, Backbone=ResNet-182025.05 | 0.9381 | 0.8758 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.9278 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8779 | |
| No DefenseAttack scenario=Blended, Backbone=ResNet-182025.05 | 0.9218 | 0.8792 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseAttack scenario=Chameleon, Backbone=ResNet-182025.05 | 0.9167 | 0.8736 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseAttack scenario=Edge-case, Backbone=ResNet-182025.05 | 0.8976 | 0.8814 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Multi-KrumAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.8842 | 0.7634 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Multi-KrumAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.8693 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.7669 | |
| FoolsGoldAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.7638 | 0.8216 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FoolsGoldAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.7603 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8111 | |
| RFAModel=ResNet92025.03 | 0.7549 | 0.8773 | 0.7067 | 0.9024 | 0.2774 | 0.0926 | 0.4767 | 0.6697 | 0.4729 | 0.3014 | 0.8127 | 0.9613 | 0.1711 | 0.0369 | 0.2254 | — | |
| FedAvgModel=ResNet92025.03 | 0.6168 | 0.8947 | 0.5156 | 0.6761 | 0.4579 | 0.3124 | 0.5621 | 0.7042 | 0.4062 | 0.2792 | 0.4489 | 0.794 | 0.5041 | 0.196 | 0.3593 | — | |
| MMModel=ResNet92025.03 | 0.6108 | 0.8915 | 0.4119 | 0.9388 | 0.5388 | 0.0601 | 0.5224 | 0.513 | 0.4354 | 0.4508 | 0.4392 | 0.8392 | 0.5112 | 0.1511 | 0.3579 | — | |
| MKrumModel=ResNet92025.03 | 0.5659 | 0.8702 | 0.811 | 0.9747 | 0.1811 | 0.0251 | 0.0217 | 0.0433 | 0.8389 | 0.791 | 0.6528 | 0.8918 | 0.3181 | 0.1001 | 0.3757 | — | |
| DeepSightAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.4864 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8105 | |
| DeepSightAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.4526 | 0.8175 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FoolsgoldModel=ResNet92025.03 | 0.426 | 0.8949 | 0.6914 | 0.6884 | 0.2964 | 0.301 | 0.5118 | 0.6073 | 0.4483 | 0.3608 | 0.0291 | 0.0282 | 0.8527 | 0.8476 | 0.5178 | — | |
| FLAMEAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.1507 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8442 | |
| FLAMEAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.1247 | 0.8526 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLTrustAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.1102 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8573 | |
| FLTrustAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.0826 | 0.8674 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LockdownModel=ResNet92025.03 | 0.0691 | 0.8856 | 0.0631 | 0.1082 | 0.8188 | 0.795 | 0.1163 | 0.0603 | 0.7882 | 0.7577 | 0.034 | 0.0327 | 0.8273 | 0.8314 | 0.8031 | — | |
| FeRAAttack scenario=IBA, Backbone=ResNet-182025.05 | 0.0508 | 0.8695 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RLRAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.0346 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.7668 | |
| FeRAAttack scenario=Blended, Backbone=ResNet-182025.05 | 0.0335 | 0.8724 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RLRModel=ResNet92025.03 | 0.0289 | 0.7916 | 0.0232 | 0.02 | 0.7672 | 0.7333 | 0.0301 | 0.0304 | 0.7709 | 0.7713 | 0.0312 | 0.0387 | 0.7398 | 0.7329 | 0.7532 | — | |
| AlignInsModel=ResNet92025.03 | 0.0221 | 0.8864 | 0.0191 | 0.0221 | 0.8603 | 0.8557 | 0.0213 | 0.0214 | 0.8577 | 0.8588 | 0.0266 | 0.022 | 0.8546 | 0.8531 | 0.8567 | — | |
| RLRAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.0218 | 0.7624 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FeRAAttack scenario=Average across attacks, Backbone=ResNet-182025.05 | 0.0217 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8742 | |
| FedAvg*Model=ResNet92025.03 | 0.0206 | 0.8947 | 0.0206 | 0.0206 | 0.856 | 0.856 | 0.0206 | 0.0206 | 0.856 | 0.856 | 0.0206 | 0.0206 | 0.856 | 0.856 | 0.856 | — | |
| FeRAAttack scenario=Chameleon, Backbone=ResNet-182025.05 | 0.0147 | 0.8785 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FeRAAttack scenario=BadNet, Backbone=ResNet-182025.05 | 0.0126 | 0.8793 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FeRAAttack scenario=Edge-case, Backbone=ResNet-182025.05 | 0.0094 | 0.8763 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FeRAAttack scenario=Neurotoxin, Backbone=ResNet-182025.05 | 0.0093 | 0.8654 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| No DefenseAttack scenario=No Attack, Backbone=ResNet-182025.05 | 0 | 0.8871 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FeRAAttack scenario=No Attack, Backbone=ResNet-182025.05 | 0 | 0.8842 | — | — | — | — | — | — | — | — | — | — | — | — | — | — |