Professional Knowledge Reasoning on MMLU-Pro
44.8MMLU-Pro Chemistry AccuracyRLKV
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RLKVKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 44.8 | 47.6 | 22.8 | 42.8 | |
| RLKVKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 44.4 | 49.3 | 23.6 | 40.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 43.2 | 48.3 | 24.2 | 40.6 | |
| R-KVKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 41 | 47.6 | 22 | 38.8 | |
| KVZipKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 40.6 | 47.8 | 16.6 | 38.4 | |
| KVZipKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 38.6 | 40.5 | 15.6 | 36.2 | |
| RLKVKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 37.8 | 43.2 | 20.6 | 37.4 | |
| DuoAttentionKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 36.4 | 46.1 | 19.8 | 38.6 | |
| R-KVKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 31.2 | 44.4 | 15.8 | 36.4 | |
| DuoAttentionKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 30.4 | 34.4 | 13 | 29.4 | |
| H2OKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 26.8 | 42 | 21.2 | 27 | |
| R-KVKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 21.6 | 31.2 | 13.8 | 26.8 | |
| H2OKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 7.8 | 27.6 | 16.2 | 13 | |
| DuoAttentionKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 3.8 | 9.8 | 3.6 | 6.6 | |
| R-KVKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 3.4 | 9.5 | 1.2 | 5.4 | |
| RLKVKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 2.2 | 5.1 | 4.2 | 4.4 | |
| H2OKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 1.6 | 6.6 | 3.8 | 3.6 | |
| KVZipKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 0.2 | 2.9 | 0.8 | 1.2 | |
| H2OKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 0 | 0 | 0 | 0 | |
| KVZipKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 0 | 0.2 | 0.8 | 0 |