Code Generation on HumanEval (HEval)
90.9HEvalQwen2.5-Coder
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2.5-CoderModel Size=14B, GPU Hours=∼1.8M2025.05 | 90.9 | |
| InfiFPOModel Size=14B, GPU Hours=582025.05 | 87.8 | |
| SFTModel Size=14B, GPU Hours=152025.05 | 86.59 | |
| SFT-WRPOModel Size=14B, GPU Hours=572025.05 | 86.59 | |
| InfiFPO*Model Size=14B, GPU Hours=55, Subset Source Models=true2025.05 | 85.98 | |
| SFT-IPOModel Size=14B, GPU Hours=502025.05 | 85.37 | |
| SFT-DPOModel Size=14B, GPU Hours=502025.05 | 84.76 | |
| Mistral-SmallModel Size=24B, GPU Hours=∼1.6M2025.05 | 84.15 | |
| FuseChat*Model Size=14B, GPU Hours=650, Subset Source Models=true2025.05 | 84.15 | |
| FuseLLM*Model Size=14B, GPU Hours=225, Subset Source Models=true2025.05 | 84 | |
| Phi-4Model Size=14B, GPU Hours=∼1.0M2025.05 | 83.54 | |
| Qwen2.5-InstructModel Size=14B, GPU Hours=∼1.8M2025.05 | 83.54 | |
| InfiFusion*Model Size=14B, GPU Hours=160, Subset Source Models=true2025.05 | 83.54 | |
| Gemma-3-InstructModel Size=12B2025.05 | 82.32 | |
| Qwen2.5-MathModel Size=7B, GPU Hours=∼0.5M2025.05 | 46.34 |