Efficiency Benchmarking on DeepSeek-R1-Distill-Llama-8B (8K Generation Length)
8Max Batch SizeSnapKV
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SnapKVBudget=Fixed - 10242026.06 | 8 | 221.58 | 3.15 | 295.77 | |
| SnapKVBudget=Fixed - 15362026.06 | 8 | 186.41 | 2.65 | 351.57 | |
| SnapKVBudget=Fixed - 20482026.06 | 8 | 163.26 | 2.32 | 401.41 | |
| R-KVBudget=Fixed - 10242026.06 | 8 | 217.35 | 3.09 | 301.52 | |
| R-KVBudget=Fixed - 15362026.06 | 8 | 183.54 | 2.61 | 357.07 | |
| R-KVBudget=Fixed - 20482026.06 | 8 | 161.39 | 2.29 | 406.07 | |
| ReasonAllocBudget=Dynamic - 10242026.06 | 8 | 218.78 | 3.11 | 299.56 | |
| ReasonAllocBudget=Dynamic - 15362026.06 | 8 | 184.62 | 2.62 | 354.97 | |
| ReasonAllocBudget=Dynamic - 20482026.06 | 8 | 161.91 | 2.3 | 404.77 | |
| FullKVBudget=FullKV2026.06 | 4 | 70.45 | 1 | 465.1 |