Code Generation on HumanEval+ and LiveCodeBench
84.1Eval+ ScoreBaseline
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaselineModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=None2025.10 | 84.1 | 43.1 | 63.6 | |
| REAPModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.10 | 84 | 44 | 64 | |
| REAPModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.10 | 83.1 | 41.6 | 62.4 | |
| BaselineModel=Kimi-K2-Instruct-W4A16, Compression=None2025.10 | 82.8 | 43.4 | 63.1 | |
| EANModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.10 | 82.7 | 41.9 | 62.3 | |
| REAPModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.10 | 82.2 | 41.5 | 61.9 | |
| REAPModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.10 | 81.9 | 42.9 | 62.4 | |
| EANModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.10 | 77.9 | 37.9 | 57.9 | |
| EANModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.10 | 77.7 | 38.2 | 58 | |
| FrequencyModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=25%2025.10 | 73.7 | 29.6 | 51.6 | |
| EANModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.10 | 72.2 | 25.3 | 48.7 | |
| FrequencyModel=Kimi-K2-Instruct-W4A16, Compression=25%2025.10 | 48.6 | 8.2 | 28.4 | |
| FrequencyModel=Kimi-K2-Instruct-W4A16, Compression=50%2025.10 | 11.2 | 0 | 5.6 | |
| FrequencyModel=Qwen3-Coder-480B-A35B-Instruct-FP8, Compression=50%2025.10 | 0.7 | 1.2 | 1 |