Code Generation on CodeAlpacaPy
3.23SpeedupDEER
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DEERBackbone=Qwen3-30B-A3B, temperature=0.6, KV cache=true2025.12 | 3.23 | 3.82 | |
| DEERBase Model=Qwen3-30B-A3B, Temperature=0, KV Cache=true2025.12 | 3.08 | 4.04 | |
| DEERBackbone=Qwen3-14B, temperature=0.6, KV cache=true2025.12 | 2.8 | 3.69 | |
| DEERBase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 2.58 | 4.21 | |
| DEERBackbone=Qwen3-4B, temperature=0.6, KV cache=true2025.12 | 2.51 | 4.1 | |
| DEERBase Model=Qwen2-7B, Temperature=0, KV Cache=true2025.12 | 2.5 | 3.69 | |
| DEERBase Model=Qwen3-14B, Temperature=0, KV Cache=true2025.12 | 2.43 | 3.98 | |
| EAGLE3Base Model=Qwen2-7B, Temperature=0, KV Cache=true2025.12 | 2.4 | 3.27 | |
| DEERBackbone=Qwen3-8B, temperature=0.6, KV cache=true2025.12 | 2.4 | 3.81 | |
| DEERBase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 2.35 | 4.06 | |
| DEERBackbone=Qwen2-7B, temperature=0.6, KV cache=true2025.12 | 2.27 | 3.5 | |
| EAGLE3Base Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 2.21 | 3.25 | |
| EAGLE3Base Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 2.2 | 3.11 | |
| EAGLE3Backbone=Qwen2-7B, temperature=0.6, KV cache=true2025.12 | 2.18 | 3.23 | |
| HydraBase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 2.17 | 2.5 | |
| EAGLE3Base Model=Qwen3-14B, Temperature=0, KV Cache=true2025.12 | 2.13 | 3.45 | |
| EAGLE3Base Model=Qwen3-30B-A3B, Temperature=0, KV Cache=true2025.12 | 2.06 | 2.89 | |
| HydraBase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 2.02 | 2.53 | |
| EAGLE3Backbone=Qwen3-14B, temperature=0.6, KV cache=true2025.12 | 2.01 | 2.44 | |
| EAGLE3Backbone=Qwen3-30B-A3B, temperature=0.6, KV cache=true2025.12 | 1.91 | 2.31 | |
| EAGLE3Backbone=Qwen3-4B, temperature=0.6, KV cache=true2025.12 | 1.86 | 2.43 | |
| EAGLE3Backbone=Qwen3-8B, temperature=0.6, KV cache=true2025.12 | 1.55 | 2.58 | |
| MEDUSABase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.12 | 1.33 | 1.98 | |
| MEDUSABase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.12 | 1.22 | 1.85 |