Multiple-choice Question Answering on MMLU-Pro Chem.
72.8AccuracyDuoAttention
Evaluation Results
| Method | Links | |
|---|---|---|
| DuoAttentionKV Cache Budget Sparsity=0.22025.10 | 72.8 | |
| RLKVKV Cache Budget Sparsity=0.42025.10 | 72.2 | |
| KVZipKV Cache Budget Sparsity=0.42025.10 | 70.8 | |
| KVZipKV Cache Budget Sparsity=0.22025.10 | 69 | |
| R-KVKV Cache Budget Sparsity=0.22025.10 | 65 | |
| RLKVKV Cache Budget Sparsity=0.22025.10 | 64.6 | |
| RLKVKV Cache Budget Sparsity=0.62025.10 | 62.6 | |
| DuoAttentionKV Cache Budget Sparsity=0.42025.10 | 62.4 | |
| KVZipKV Cache Budget Sparsity=0.62025.10 | 56 | |
| RLKVKV Cache Budget Sparsity=0.22025.10 | 55 | |
| KVZipKV Cache Budget Sparsity=0.22025.10 | 51 | |
| RLKVKV Cache Budget Sparsity=0.42025.10 | 50.8 | |
| R-KVKV Cache Budget Sparsity=0.42025.10 | 49.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.22025.10 | 47.4 | |
| DuoAttentionKV Cache Budget Sparsity=0.42025.10 | 44.8 | |
| R-KVKV Cache Budget Sparsity=0.22025.10 | 43.4 | |
| KVZipKV Cache Budget Sparsity=0.42025.10 | 41.2 | |
| RLKVKV Cache Budget Sparsity=0.62025.10 | 36 | |
| R-KVKV Cache Budget Sparsity=0.42025.10 | 32.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.62025.10 | 32.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.62025.10 | 24.8 | |
| KVZipKV Cache Budget Sparsity=0.62025.10 | 22.8 | |
| H2OKV Cache Budget Sparsity=0.22025.10 | 21.6 | |
| R-KVKV Cache Budget Sparsity=0.62025.10 | 19.8 | |
| H2OKV Cache Budget Sparsity=0.22025.10 | 18.8 | |
| R-KVKV Cache Budget Sparsity=0.62025.10 | 17.4 | |
| H2OKV Cache Budget Sparsity=0.42025.10 | 9 | |
| RLKVKV Cache Budget Sparsity=0.82025.10 | 7.4 | |
| H2OKV Cache Budget Sparsity=0.42025.10 | 7.2 | |
| H2OKV Cache Budget Sparsity=0.62025.10 | 3 | |
| H2OKV Cache Budget Sparsity=0.62025.10 | 2.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.82025.10 | 2.2 | |
| DuoAttentionKV Cache Budget Sparsity=0.82025.10 | 1.4 | |
| R-KVKV Cache Budget Sparsity=0.82025.10 | 1 | |
| R-KVKV Cache Budget Sparsity=0.82025.10 | 0.8 | |
| H2OKV Cache Budget Sparsity=0.82025.10 | 0.4 | |
| KVZipKV Cache Budget Sparsity=0.82025.10 | 0.4 | |
| RLKVKV Cache Budget Sparsity=0.82025.10 | 0.4 | |
| KVZipKV Cache Budget Sparsity=0.82025.10 | 0.2 | |
| H2OKV Cache Budget Sparsity=0.82025.10 | 0 |