Natural Language Understanding on GLUE adversarial (test)
72RTE AccuracyPABEE
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PABEEBackbone=ALBERT-base, Attack=WAFFLE (A2T)2023.10 | 72 | 0.17 | 69 | 0.18 | 73 | 0.21 | 82 | 0.23 | 79 | 0.27 | 85 | 0.29 | 60 | 0.19 | |
| Past-FutureBackbone=ALBERT-base, Attack=WAFFLE (A2T)2023.10 | 64 | 0.36 | 67 | 0.26 | 72 | 0.29 | 83 | 0.33 | 79 | 0.39 | — | — | — | — | |
| Past-FutureBackbone=ALBERT-base, Attack=A2T2023.10 | 58 | 0.49 | 59 | 0.32 | 58 | 0.31 | 74 | 0.35 | 64 | 0.43 | — | — | — | — | |
| DeeBERTBackbone=BERT-base, Attack=WAFFLE (A2T)2023.10 | 57 | 0.19 | — | — | 75 | 0.17 | 78 | 0.19 | 83 | 0.3 | — | — | 73 | 0.24 | |
| PABEEBackbone=ALBERT-base, Attack=A2T2023.10 | 57 | 0.22 | 52 | 0.21 | 63 | 0.26 | 71 | 0.27 | 61 | 0.31 | 76 | 0.32 | 38 | 0.23 | |
| PABEEBackbone=ALBERT-base, Attack=WAFFLE (TF)2023.10 | 57 | 0.09 | 38 | 0.05 | 47 | 0.08 | 51 | 0.06 | 67 | 0.17 | 50 | 0.08 | 48 | 0.08 | |
| DeeBERTBackbone=BERT-base, Attack=A2T2023.10 | 52 | 0.32 | — | — | 75 | 0.33 | 81 | 0.35 | 74 | 0.41 | — | — | 66 | 0.29 | |
| DeeBERTBackbone=BERT-base, Attack=WAFFLE (TF)2023.10 | 51 | 0.11 | — | — | 61 | 0.09 | 62 | 0.1 | 69 | 0.22 | — | — | 70 | 0.13 | |
| Past-FutureBackbone=ALBERT-base, Attack=WAFFLE (TF)2023.10 | 51 | 0.17 | 45 | 0.05 | 42 | 0.15 | 58 | 0.07 | 64 | 0.25 | — | — | — | — | |
| DeeBERTBackbone=BERT-base, Attack=TF2023.10 | 48 | 0.32 | — | — | 75 | 0.32 | 78 | 0.33 | 67 | 0.4 | — | — | 57 | 0.2 | |
| Past-FutureBackbone=ALBERT-base, Attack=TF2023.10 | 41 | 0.46 | 42 | 0.24 | 36 | 0.24 | 52 | 0.19 | 50 | 0.35 | — | — | — | — | |
| PABEEBackbone=ALBERT-base, Attack=TF2023.10 | 34 | 0.22 | 25 | 0.17 | 37 | 0.21 | 33 | 0.18 | 31 | 0.27 | 22 | 0.26 | 5 | 0.23 |