Question Answering on SQuAD (EM, F1, BOPs)
80.08EM ScoreBaseline
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BaselineSparsity=0%2025.02 | 80.08 | 88.5 | 13.57 | 100 | |
| GETASparsity=10%2025.02 | 78.26 | 86.06 | 2.63 | 19.4 | |
| GETASparsity=30%2025.02 | 77.28 | 85.7 | 2.29 | 16.9 | |
| GETASparsity=50%2025.02 | 76.74 | 85.87 | 1.96 | 14.4 | |
| GETASparsity=70%2025.02 | 75.88 | 84.74 | 1.62 | 11.9 | |
| OTO followed up 8-bit PTQSparsity=10%2025.02 | 73.87 | 83.43 | 3.17 | 23.4 | |
| OTO followed up 8-bit PTQSparsity=30%2025.02 | 72.95 | 83.31 | 2.71 | 20 | |
| OTO followed up 8-bit PTQSparsity=50%2025.02 | 72.71 | 83.3 | 2.26 | 16.7 | |
| OTO followed up 8-bit PTQSparsity=70%2025.02 | 71.24 | 82.57 | 1.8 | 13.3 |