Natural Language Understanding on GLUE (Comprehensive 9-Task Evaluation)
95.4SST-2 AccuracyRoBERTaLarge-24L
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| RoBERTaLarge-24L#Bits (W-A)=32-32, Size (MB)=1362, GFLOPs=23.842024.12 | 95.4 | 66.4 | 91.6 | 86.6 | 89 | 89.6 | 92.3 | 94.2 | 90.7 | 88.4 | |
| ALBERTLarge-24L#Bits (W-A)=32-32, Size (MB)=69, GFLOPs=24.302024.12 | 95.2 | 60.1 | 90.4 | 83 | 86 | 86.1 | 91.4 | 91.6 | 89.6 | 85.9 | |
| RoBERTaBase-12L#Bits (W-A)=32-32, Size (MB)=479, GFLOPs=6.732024.12 | 94.8 | 63.6 | 90.8 | 77.5 | 87.5 | 87.2 | 90.9 | 92.7 | 90.3 | 86.1 | |
| BERTLarge-24L#Bits (W-A)=32-32, Size (MB)=1285, GFLOPs=23.452024.12 | 93.5 | 61.6 | 90.1 | 72.9 | 86.2 | 86 | 90.4 | 92.2 | 90.1 | 84.8 | |
| BERTBase-12L#Bits (W-A)=32-32, Size (MB)=418, GFLOPs=6.622024.12 | 92.9 | 56.5 | 87.6 | 69 | 84.6 | 84.9 | 89.4 | 91.2 | 89.6 | 82.9 | |
| ALBERTBase-12L#Bits (W-A)=32-32, Size (MB)=46, GFLOPs=6.862024.12 | 92.8 | 56.8 | 90.5 | 78.3 | 84.9 | 85.6 | 90.7 | 91.4 | 89.2 | 84.5 | |
| BEExformer#Bits (W-A)=1-1, Size (MB)=14.26, GFLOPs=0.122024.12 | 92.32 | 60.3 | 86.47 | 71.11 | 83.71 | 84.29 | 86.6 | 89.18 | 88.09 | 82.45 | |
| ALBERTLarge-6L#Bits (W-A)=32-32, Size (MB)=69, GFLOPs=6.082024.12 | 92.2 | 51.7 | 86.5 | 66.4 | 82.2 | 82.9 | 89.4 | 89.4 | 88.6 | 81 | |
| RoBERTaBase-6L#Bits (W-A)=32-32, Size (MB)=314, GFLOPs=3.362024.12 | 92.1 | 44.4 | 87.9 | 60.6 | 84.2 | 84.6 | 89 | 90.5 | 89.8 | 80.3 | |
| BERTBase-6L#Bits (W-A)=32-32, Size (MB)=257, GFLOPs=3.312024.12 | 90.9 | 44.6 | 84.9 | 65.7 | 81.4 | 81.4 | 88.1 | 87.4 | 88.7 | 79.2 | |
| ALBERTBase-6L#Bits (W-A)=32-32, Size (MB)=46, GFLOPs=3.442024.12 | 90.8 | 52.4 | 89 | 70.4 | 82.6 | 82.2 | 89.6 | 89.8 | 88.7 | 81.7 | |
| RoBERTaLarge-6L#Bits (W-A)=32-32, Size (MB)=495, GFLOPs=5.962024.12 | 90.1 | 43.3 | 80 | 54.9 | 80.4 | 80.9 | 80.5 | 86.1 | 88.9 | 76.1 | |
| BERTLarge-6L#Bits (W-A)=32-32, Size (MB)=414, GFLOPs=5.862024.12 | 89.7 | 20.2 | 76.4 | 58.5 | 76.5 | 76.5 | 77.3 | 84.3 | 87.3 | 71.9 |