Natural Language Understanding on GLUE 1.0 (dev)
95.3SST-2 (Acc)RoBERTa
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| RoBERTaScale=large2022.03 | 95.3 | — | — | — | — | — | — | 89 | |
| XLNetScale=large2022.03 | 94.4 | 65.2 | 90 | — | — | — | — | 88.4 | |
| BERTScale=large, variant=wwm, reproduced=true2022.03 | 93.4 | 63.1 | 87.2 | — | — | — | — | 87.3 | |
| PERTScale=large2022.03 | 93.4 | 65.7 | 87.3 | — | — | — | — | 87.6 | |
| BERTScale=large2022.03 | 93.2 | 60.6 | 88 | — | — | — | — | 86.6 | |
| BERTScale=base2022.03 | 92.7 | 60.6 | 86.7 | — | — | — | — | 84.4 | |
| BERTScale=base, variant=+2022.03 | 92.6 | 59.3 | 86 | — | — | — | — | 84.4 | |
| XLNetScale=base2022.03 | 92.6 | — | — | — | — | — | — | 85.8 | |
| RoBERTaScale=base2022.03 | 92.5 | — | — | — | — | — | — | 84.7 | |
| BERT baselineArchitecture=BERT-Base, Precision=FP32, Quantization Method=None2019.10 | 92.36 | 58.48 | 90 | 90.3 | 87.84 | 69.7 | 89.62 | — | |
| Q8BERTArchitecture=BERT-Base, Precision=8bit, Quantization Method=Quantization-Aware Training (QAT)2019.10 | 92.24 | 58.48 | 89.56 | 90.62 | 87.96 | 68.78 | 89.04 | — | |
| PERTScale=base2022.03 | 92 | 61.2 | 87.5 | — | — | — | — | 84.5 | |
| DQ BERTArchitecture=BERT-Base, Precision=8bit, Quantization Method=Dynamic Quantization (DQ)2019.10 | 91.04 | 56.74 | 87.88 | 89.34 | 84.98 | 63.32 | 87.66 | — | |
| ALBERTScale=large2022.03 | 90.6 | — | — | — | — | — | — | 83.8 | |
| ALBERTScale=base2022.03 | 89.4 | — | — | — | — | — | — | 81.9 | |
| BERT baselineArchitecture=BERT-Large, Precision=FP32, Quantization Method=None2019.10 | — | — | 90.86 | 91.66 | — | — | 90.34 | — | |
| DQ BERTArchitecture=BERT-Large, Precision=8bit, Quantization Method=Dynamic Quantization (DQ)2019.10 | — | — | 88.18 | 88.38 | — | — | 83.04 | — | |
| Q8BERTArchitecture=BERT-Large, Precision=8bit, Quantization Method=Quantization-Aware Training (QAT)2019.10 | — | — | 90.9 | 91.74 | — | — | 90.12 | — |