Code Generation on LiveCodeBench Jan-Apr 2025
47.25Accuracy (pass@1)BF16
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16Model=DeepSeek-R1-Distill-Qwen-32B, Bit-width=BF162025.12 | 47.25 | |
| BF16Model=DeepSeek-R1-Distill-Qwen-14B, Bit-width=BF162025.12 | 45.05 | |
| MixKVQModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=C2.32025.12 | 45.05 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV42025.12 | 44.51 | |
| KVQuantModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV42025.12 | 43.96 | |
| MixKVQModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=C2.32025.12 | 43.41 | |
| RotateKVModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV42025.12 | 42.86 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV42025.12 | 42.31 | |
| RotateKVModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV42025.12 | 42.31 | |
| KVQuantModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV42025.12 | 41.76 | |
| KVTunerModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=C2.912025.12 | 41.76 | |
| KVTunerModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=C2.902025.12 | 40.11 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV22025.12 | 39.56 | |
| BF16Model=DeepSeek-R1-Distill-Llama-8B, Bit-width=BF162025.12 | 34.62 | |
| MixKVQModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=C2.72025.12 | 31.87 | |
| KVTunerModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=C3.252025.12 | 31.32 | |
| KIVIModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV22025.12 | 31.32 | |
| RotateKVModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV42025.12 | 30.22 | |
| KIVIModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV42025.12 | 28.57 | |
| KVQuantModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV42025.12 | 27.47 | |
| KVQuantModel=DeepSeek-R1-Distill-Qwen-32B, Bit-width=KV22025.12 | 20.33 | |
| KIVIModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV22025.12 | 17.58 | |
| KVQuantModel=DeepSeek-R1-Distill-Qwen-14B, Bit-width=KV22025.12 | 14.29 | |
| KVQuantModel=DeepSeek-R1-Distill-Llama-8B, Bit-width=KV22025.12 | 12.64 |