Code Generation on SWE-bench Lite
70GF PrecisionSnapKV
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SnapKVModel=Qwen-Coder, KV cache capacity ratio (Cap.)=0.62026.04 | 70 | 70 | 70 | 70 | 100 | 0.718 | |
| No Comp.Model=Qwen-Coder, KV cache capacity ratio (Cap.)=1.02026.04 | 62.5 | 65 | 63.7 | 62.5 | 100 | 0.666 | |
| CodeCompModel=Qwen-Coder, KV cache capacity ratio (Cap.)=0.62026.04 | 61.3 | 61.3 | 61.3 | 61.3 | 100 | 0.683 | |
| SnapKVModel=Qwen-Coder, KV cache capacity ratio (Cap.)=0.42026.04 | 60 | 60 | 60 | 60 | 95 | 0.709 | |
| CodeCompModel=Qwen-Coder, KV cache capacity ratio (Cap.)=0.42026.04 | 54.4 | 55 | 54.7 | 54.4 | 98.8 | 0.686 | |
| ParallelCompModel=Qwen-Coder, KV cache capacity ratio (Cap.)=0.42026.04 | 36.9 | 37.5 | 37.2 | 36.9 | 66.3 | 0.8 | |
| SnapKVModel=DS-Coder, KV cache capacity ratio (Cap.)=0.62026.04 | 35 | 35 | 35 | 35 | 80 | 0.77 | |
| No Comp.Model=DS-Coder, KV cache capacity ratio (Cap.)=1.02026.04 | 33.8 | 33.8 | 33.8 | 33.8 | 87.5 | 0.74 | |
| ParallelCompModel=Qwen-Coder, KV cache capacity ratio (Cap.)=0.62026.04 | 29.4 | 30 | 29.7 | 29.4 | 66.3 | 0.809 | |
| CodeCompModel=DS-Coder, KV cache capacity ratio (Cap.)=0.42026.04 | 25 | 25 | 25 | 25 | 100 | 0.743 | |
| CodeCompModel=DS-Coder, KV cache capacity ratio (Cap.)=0.62026.04 | 25 | 25 | 25 | 25 | 88.8 | 0.766 | |
| SnapKVModel=DS-Coder, KV cache capacity ratio (Cap.)=0.42026.04 | 20 | 20 | 20 | 20 | 90 | 0.758 | |
| ParallelCompModel=DS-Coder, KV cache capacity ratio (Cap.)=0.62026.04 | 3.8 | 3.8 | 3.8 | 3.8 | 22.5 | 0.869 | |
| ParallelCompModel=DS-Coder, KV cache capacity ratio (Cap.)=0.42026.04 | 1.9 | 2.5 | 2.1 | 1.9 | 35 | 0.824 |