Question Answering on SQuAD (val)
91.67F1 ScoreOurs 4:8-T
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ours 4:8-TModel=Bert-Large, SU=66%2021.02 | 91.67 | — | — | |
| ASP 2:4Model=Bert-Large, SU=33%2021.02 | 91.5 | — | — | |
| BaselineModel=Bert-Large, SU=0%2021.02 | 91.1 | — | — | |
| Full-FTTraining Epochs=32023.05 | 90.7 | — | — | |
| MobileBERT (Sun et al., 2020)Speedup=2.3x, Architecture Design=Specialized architecture design2022.04 | 90.3 | — | — | |
| HiFiTraining Epochs=32023.05 | 89.8 | — | — | |
| LoRATraining Epochs=32023.05 | 89.2 | — | — | |
| CoFiSpeedup=2.0x, Model Size=34M, Sparsity=60%2022.04 | 89.1 | — | — | |
| Diff-PruningTraining Epochs=32023.05 | 89 | — | — | |
| Child-TuningTraining Epochs=32023.05 | 88.5 | — | — | |
| Baseline# of Parameters=100%2021.07 | 88.3 | 81 | 1 | |
| AdapterTraining Epochs=32023.05 | 88 | — | — | |
| Prefix-tuningTraining Epochs=32023.05 | 87.7 | — | — | |
| AutoTinyBERT (Yin et al., 2021)Speedup=4.3x, Base Model=ELECTRA-base2022.04 | 87.6 | — | — | |
| CompacterTraining Epochs=32023.05 | 87.1 | — | — | |
| CoFiSpeedup=4.6x, Model Size=13M, Sparsity=85%2022.04 | 86.4 | — | — | |
| Wang et al. (2020b)Speedup=1.5x2022.04 | 85.4 | — | — | |
| OTO# of Parameters=91.0%2021.07 | 84.1 | 75 | 1.1 | |
| OTO# of Parameters=76.2%2021.07 | 82.1 | 72.3 | 1.2 | |
| BitFitTraining Epochs=32023.05 | 82.1 | — | — | |
| ProxSSI# of Parameters=83.4%2021.07 | 82 | 72.3 | 1 | |
| OTO# of Parameters=66.7%2021.07 | 82 | 71.9 | 1.3 | |
| MvP# of Parameters=10.0%2021.07 | 81.7 | 71.9 | 1 | |
| OTO# of Parameters=53.3%2021.07 | 81.5 | 71.4 | 1.5 | |
| OTO# of Parameters=40.0%2021.07 | 81.1 | 70.9 | 1.8 | |
| MaP# of Parameters=10.0%2021.07 | 78.5 | 67.7 | 1 |