Code Generation on MBPP (Performance %)
83Performance (%)DuoAttention
Evaluation Results
| Method | Links | |
|---|---|---|
| DuoAttentionKV Cache Budget Sparsity=0.22025.10 | 83 | |
| RLKVKV Cache Budget Sparsity=0.22025.10 | 82.4 | |
| RLKVKV Cache Budget Sparsity=0.42025.10 | 81 | |
| KVZipKV Cache Budget Sparsity=0.22025.10 | 78.2 | |
| DuoAttentionKV Cache Budget Sparsity=0.42025.10 | 76.4 | |
| KVZipKV Cache Budget Sparsity=0.42025.10 | 74.4 | |
| R-KVKV Cache Budget Sparsity=0.22025.10 | 67 | |
| RLKVKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 63.8 | |
| KVZipKV Cache Budget Sparsity=0.62025.10 | 63.8 | |
| RLKVKV Cache Budget Sparsity=0.22025.10 | 63.2 | |
| RLKVKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 62.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 62 | |
| DuoAttentionKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 60.6 | |
| DuoAttentionKV Cache Budget Sparsity=0.22025.10 | 60.4 | |
| KVZipKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 59.8 | |
| KVZipKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 57.8 | |
| RLKVKV Cache Budget Sparsity=0.42025.10 | 56.8 | |
| KVZipKV Cache Budget Sparsity=0.22025.10 | 56.6 | |
| RLKVKV Cache Budget Sparsity=0.62025.10 | 55.2 | |
| DuoAttentionKV Cache Budget Sparsity=0.42025.10 | 55 | |
| RLKVKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 51.8 | |
| KVZipKV Cache Budget Sparsity=0.42025.10 | 50 | |
| R-KVKV Cache Budget Sparsity=0.42025.10 | 46.6 | |
| R-KVKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 46.4 | |
| R-KVKV Cache Budget Sparsity=0.22025.10 | 41.4 | |
| DuoAttentionKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 40.4 | |
| RLKVKV Cache Budget Sparsity=0.62025.10 | 40.2 | |
| DuoAttentionKV Cache Budget Sparsity=0.62025.10 | 39.8 | |
| KVZipKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 39.6 | |
| KVZipKV Cache Budget Sparsity=0.62025.10 | 39.2 | |
| DuoAttentionKV Cache Budget Sparsity=0.62025.10 | 35 | |
| R-KVKV Cache Budget Sparsity=0.42025.10 | 31 | |
| R-KVKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 29.4 | |
| R-KVKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 14.6 | |
| KVZipKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 12.6 | |
| H2OKV Cache Budget Sparsity=0.22025.10 | 12.6 | |
| R-KVKV Cache Budget Sparsity=0.62025.10 | 12.2 | |
| R-KVKV Cache Budget Sparsity=0.62025.10 | 10.8 | |
| DuoAttentionKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 7.4 | |
| H2OKV Cache Budget Sparsity=0.2, Backbone=Llama-3.1-8B-R12025.10 | 7.2 | |
| RLKVKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 6 | |
| H2OKV Cache Budget Sparsity=0.22025.10 | 5.2 | |
| H2OKV Cache Budget Sparsity=0.42025.10 | 1.6 | |
| R-KVKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 1 | |
| DuoAttentionKV Cache Budget Sparsity=0.82025.10 | 1 | |
| DuoAttentionKV Cache Budget Sparsity=0.82025.10 | 1 | |
| RLKVKV Cache Budget Sparsity=0.82025.10 | 1 | |
| H2OKV Cache Budget Sparsity=0.4, Backbone=Llama-3.1-8B-R12025.10 | 0.6 | |
| RLKVKV Cache Budget Sparsity=0.82025.10 | 0.6 | |
| R-KVKV Cache Budget Sparsity=0.82025.10 | 0.4 | |
| KVZipKV Cache Budget Sparsity=0.82025.10 | 0.4 | |
| H2OKV Cache Budget Sparsity=0.42025.10 | 0.4 | |
| KVZipKV Cache Budget Sparsity=0.82025.10 | 0.4 | |
| H2OKV Cache Budget Sparsity=0.62025.10 | 0.2 | |
| R-KVKV Cache Budget Sparsity=0.82025.10 | 0.2 | |
| H2OKV Cache Budget Sparsity=0.6, Backbone=Llama-3.1-8B-R12025.10 | 0 | |
| H2OKV Cache Budget Sparsity=0.8, Backbone=Llama-3.1-8B-R12025.10 | 0 | |
| H2OKV Cache Budget Sparsity=0.82025.10 | 0 | |
| H2OKV Cache Budget Sparsity=0.62025.10 | 0 | |
| H2OKV Cache Budget Sparsity=0.82025.10 | 0 |