Repository-level Code Completion on RepoBench-P 128K context
60ScoreFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=Qwen2.5-32B2026.03 | 60 | 2,400 | |
| TTKVModel=Qwen2.5-32B2026.03 | 59.9 | 560 | |
| FP16Model=DeepSeek-R1-14B2026.03 | 59.5 | 2,600 | |
| TTKVModel=DeepSeek-R1-14B2026.03 | 59.3 | 470 | |
| DiffKVModel=Qwen2.5-32B2026.03 | 58.5 | 900 | |
| ShadowKVModel=Qwen2.5-32B2026.03 | 58.3 | 880 | |
| DiffKVModel=DeepSeek-R1-14B2026.03 | 58.3 | 930 | |
| KVQuantModel=Qwen2.5-32B2026.03 | 58.2 | 980 | |
| ShadowKVModel=DeepSeek-R1-14B2026.03 | 58.1 | 900 | |
| FP16Model=Llama-3.1-70B2026.03 | 58 | 2,500 | |
| KVQuantModel=DeepSeek-R1-14B2026.03 | 58 | 1,000 | |
| TTKVModel=Llama-3.1-70B2026.03 | 57.8 | 600 | |
| DiffKVModel=Llama-3.1-70B2026.03 | 57 | 890 | |
| KIVIModel=Qwen2.5-32B2026.03 | 57 | 1,050 | |
| KVQuantModel=Llama-3.1-70B2026.03 | 56.8 | 980 | |
| ShadowKVModel=Llama-3.1-70B2026.03 | 56.8 | 860 | |
| KIVIModel=DeepSeek-R1-14B2026.03 | 56.5 | 1,070 | |
| KIVIModel=Llama-3.1-70B2026.03 | 55.8 | 1,030 |