Attention-based attribution sensitivity on STL10 96x96 (test)
0.32AttRaw ScoreSharp. Aware Min. (SAM)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Sharp. Aware Min. (SAM)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.32 | 0.27 | 0.76 | 0.76 | 0.51 | 4.9698 | |
| Implicit Curv. Reg. (ICR†)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.31 | 0.17 | 0.44 | 0.47 | 0.47 | 4.9698 | |
| Explicit Curv. Reg. (ECR)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.29 | 0.16 | 0.42 | 0.46 | 0.37 | 4.9698 | |
| Ante-hoc Act. Reg. (AAR)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.28 | 0.17 | 0.46 | 0.51 | 0.56 | 4.9698 | |
| No Regularization (Base)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.23 | 0.15 | 0.41 | 0.45 | 0.44 | 4.9698 | |
| Adv. Trainig Reg. (ATR)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.21 | 0.12 | 0.35 | 0.38 | 0.37 | 4.9698 | |
| Post-hoc Act. Reg. (PAR)Backbone=ViT-Tiny, Attention Type=Softmax Attn., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.19 | 0.13 | 0.41 | 0.46 | 0.55 | 4.9698 | |
| No Regularization (Base)Backbone=ViT-Tiny, Attention Type=Unnorm. Kern., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.04 | 0.03 | 0.12 | 0.08 | 0.69 | 4.9685 | |
| Explicit Curv. Reg. (ECR)Backbone=ViT-Tiny, Attention Type=Unnorm. Kern., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.02 | 0.03 | 0.11 | 0.09 | 0.7 | 4.9685 | |
| Implicit Curv. Reg. (ICR†)Backbone=ViT-Tiny, Attention Type=Unnorm. Kern., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0.01 | 0.08 | 0.61 | 0.1 | 1.22 | 4.9578 | |
| Adv. Trainig Reg. (ATR)Backbone=ViT-Tiny, Attention Type=Unnorm. Kern., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0 | 0.02 | 0.1 | 0.06 | 0.6 | 4.9684 | |
| Sharp. Aware Min. (SAM)Backbone=ViT-Tiny, Attention Type=Unnorm. Kern., Activation=GELU, Maximum attainable entropy=4.972026.05 | 0 | 0.04 | 0.17 | 0.07 | 0.61 | 4.9679 |