Code Synthesis on HumanEval
81.7pass@1GTPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GTPOBase Model=Qwen2.5-7B2025.11 | 81.7 | — | |
| GRPOBase Model=Qwen2.5-7B2025.11 | 77.4 | — | |
| Davinci CodexTraining stage=Code Finetuning, Shots=0-shot, Note=Calculation via OpenAI Codex API2022.04 | 36 | 81.7 | |
| PaLM-CoderTraining stage=Code Finetuning, Model scale=540B, Shots=0-shot2022.04 | 36 | 88.4 | |
| CodexTraining stage=Code Finetuning, Model scale=12B, Shots=0-shot2022.04 | 28.8 | 72.3 | |
| PaLMTraining stage=Pretraining only, Model scale=540B, Shots=0-shot2022.04 | 26.2 | 76.2 | |
| Echo-LoRAModel=LLaMA2-7B2026.05 | 25.78 | — | |
| Flat-LoRAModel=LLaMA2-7B2026.05 | 24.56 | — | |
| Echo-LoRAModel=LLaMA2-13B2026.05 | 15.85 | — | |
| LaMDATraining stage=Pretraining only, Model scale=137B, Shots=0-shot2022.04 | 14 | 47.3 | |
| Flat-LoRAModel=LLaMA2-13B2026.05 | 13.78 | — |