Code Generation on MBPP, CodeAlpacaPy, HumanEval, and LiveCodeBench (Composite)
4.04SpeedupDEER
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DEERBase Model=Qwen3-30B-A3B, Temperature=0, KV Cache=true2025.12 | 4.04 | 5.03 | |
| DEERBase Model=Qwen3-14B, Temperature=0, KV Cache=true2025.12 | 2.98 | 4.82 | |
| DEERBase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 2.83 | 4.61 | |
| DEERBase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 2.77 | 4.61 | |
| DEERBase Model=Qwen2-7B, Temperature=0, KV Cache=true2025.12 | 2.7 | 4.11 | |
| EAGLE3Base Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 2.48 | 3.45 | |
| EAGLE3Base Model=Qwen2-7B, Temperature=0, KV Cache=true2025.12 | 2.43 | 3.22 | |
| EAGLE3Base Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 2.4 | 3.31 | |
| EAGLE3Base Model=Qwen3-14B, Temperature=0, KV Cache=true2025.12 | 2.39 | 3.54 | |
| HydraBase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 2.25 | 2.66 | |
| HydraBase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 2.22 | 2.58 | |
| EAGLE3Base Model=Qwen3-30B-A3B, Temperature=0, KV Cache=true2025.12 | 2.21 | 3.05 | |
| MEDUSABase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 1.32 | 1.97 | |
| MEDUSABase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 1.23 | 1.94 |