Math & Science on GSM8K (Score)
88.02ScoreFull Cache
Evaluation results
| Method | Links | |
|---|---|---|
| Full CacheBackbone=Qwen2.5-7B-Instruct2025.02 | 88.02 | |
| ReFreeKVBackbone=Qwen2.5-7B-Instruct, k=1, Real Budget Utilization=90.7%2025.02 | 88.02 | |
| ReFreeKVBackbone=Llama3-8B-Instruct, k=1, Real Budget Utilization=93.2%2025.02 | 76.5 | |
| PyramidKVBackbone=Llama3-8B-Instruct, KV Cache Budget=90%2025.02 | 75.44 | |
| Full CacheBackbone=Llama3-8B-Instruct2025.02 | 75.28 | |
| Heavy Hitter Oracle (H2O)Backbone=Llama3-8B-Instruct, KV Cache Budget=90%2025.02 | 74.6 | |
| CAKEBackbone=Llama3-8B-Instruct, KV Cache Budget=90%2025.02 | 74.23 | |
| StreamingLLM (SLM)Backbone=Llama3-8B-Instruct, KV Cache Budget=90%2025.02 | 72.78 | |
| SnapKVBackbone=Llama3-8B-Instruct, KV Cache Budget=90%2025.02 | 70.2 | |
| Full CacheBackbone=Mistral-7B-Instruct2025.02 | 33.36 | |
| ReFreeKVBackbone=Mistral-7B-Instruct, k=1, Real Budget Utilization=89.6%2025.02 | 31.54 | |
| Heavy Hitter Oracle (H2O)Backbone=Mistral-7B-Instruct, KV Cache Budget=50%2025.02 | 2.5 |