List your company
ResearchFormerly WizWand

Code Generation on MBPP, CodeAlpacaPy, HumanEval, and LiveCodeBench (Composite)

4.04SpeedupDEER
1.11761.87632.6353.3937Dec 17, 2025
Updated 8mo ago

Evaluation results

MethodLinks
DEERBase Model=Qwen3-30B-A3B, Temperature=0, KV Cache=true2025.124.045.03
DEERBase Model=Qwen3-14B, Temperature=0, KV Cache=true2025.122.984.82
DEERBase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.122.834.61
DEERBase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.122.774.61
DEERBase Model=Qwen2-7B, Temperature=0, KV Cache=true2025.122.74.11
EAGLE3Base Model=Qwen3-8B, Temperature=0, KV Cache=true2025.122.483.45
EAGLE3Base Model=Qwen2-7B, Temperature=0, KV Cache=true2025.122.433.22
EAGLE3Base Model=Qwen3-4B, Temperature=0, KV Cache=true2025.122.43.31
EAGLE3Base Model=Qwen3-14B, Temperature=0, KV Cache=true2025.122.393.54
HydraBase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.122.252.66
HydraBase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.122.222.58
EAGLE3Base Model=Qwen3-30B-A3B, Temperature=0, KV Cache=true2025.122.213.05
MEDUSABase Model=Qwen3-8B, Temperature=0, KV Cache=true2025.121.321.97
MEDUSABase Model=Qwen3-4B, Temperature=0, KV Cache=true2025.121.231.94