Natural Language Understanding on GLUE (SST-2, MNLI, MRPC, CoLA, QNLI, QQP, STS-B)
91.51SST-2 AccuracyScratch
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ScratchBackbone=BERT-B, FLOPs Savings=0%, Walltime Savings=0.0%2026.03 | 91.51 | 83.28 | 84.31 | 43.34 | 89.93 | 90.64 | 85.47 | 81.21 | |
| BoTBackbone=BERT-B, FLOPs Savings=67.1%, Walltime Savings=65.3%2026.03 | 90.94 | 83.46 | 87.99 | 48.12 | 90.06 | 90.66 | 87.7 | 82.7 | |
| LiGOBackbone=BERT-B, FLOPs Savings=44.7%, Walltime Savings=43.5%2026.03 | 90.6 | 83.09 | 84.56 | 45.42 | 90.12 | 90.55 | 86.83 | 81.6 | |
| bert2BERTBackbone=BERT-B, FLOPs Savings=33.3%, Walltime Savings=32.4%2026.03 | 90.48 | 83.6 | 84.8 | 50.47 | 90.19 | 90.44 | 87.03 | 82.43 | |
| MangoBackbone=BERT-B, FLOPs Savings=44.9%, Walltime Savings=43.7%2026.03 | 90.25 | 83.39 | 84.07 | 45.35 | 89.82 | 90.64 | 87.12 | 81.52 | |
| WSBackbone=BERT-S, FLOPs Savings=33.0%, Walltime Savings=41.7%2026.03 | 89.45 | 77.67 | 77.7 | 28 | 85.14 | 88.93 | 82.15 | 75.58 | |
| BoTBackbone=BERT-S, FLOPs Savings=52.8%, Walltime Savings=66.8%2026.03 | 89.33 | 78.34 | 77.47 | 27.54 | 85.69 | 88.95 | 82.62 | 75.71 | |
| ScratchBackbone=BERT-S, FLOPs Savings=0.0%, Walltime Savings=0.0%2026.03 | 87.5 | 77.74 | 77.94 | 28.47 | 85.54 | 88.93 | 83.01 | 75.59 |