Language Understanding on MMLU (Bio, Bus, Chem, CS, Math, Phi, Phy Scores)
89.8MMLU Biology ScoreRTPurbo
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RTPurboselection_strategy=top-p2026.05 | 89.8 | 87.2 | 87.8 | 85.12 | 93.6 | 69.54 | 91 | |
| Full AttnAttention Strategy=Full Attention2026.05 | 89.2 | 87.4 | 88 | 86.1 | 93.6 | 69.94 | 90.8 | |
| RTPurboselection_strategy=top-k2026.05 | 88.8 | 86.6 | 51.4 | 50.49 | 71.6 | 51.7 | 90.6 | |
| QuestAttention Strategy=Sparse Attention2026.05 | 88.28 | 84.28 | 84.4 | 82.68 | 93.2 | 69.34 | 87.7 | |
| SnapKVAttention Strategy=Sparse Attention2026.05 | 88.14 | 81.27 | 76.32 | 80.49 | 90 | 66.4 | 84.42 |