Code Generation on HumanEval In-Domain (Accuracy)
53.05AccuracyPion
Evaluation results
| Method | Links | |
|---|---|---|
| PionBase Model=Qwen2.5-1.5B2026.05 | 53.05 | |
| AdamWBase Model=Qwen2.5-1.5B2026.05 | 51.83 | |
| MuonBase Model=Qwen2.5-1.5B2026.05 | 50 | |
| PionBase Model=Llama3.2-3B2026.05 | 47.19 | |
| AdamWBase Model=Llama3.2-3B2026.05 | 46.95 | |
| MuonBase Model=Llama3.2-3B2026.05 | 46.34 | |
| BaseBase Model=Qwen2.5-1.5B2026.05 | 35.98 | |
| BaseBase Model=Llama3.2-3B2026.05 | 26.22 |