Code Generation on HumanEval+ (avg@32)
85.21Average Pass Rate @32Code-A1
Evaluation Results
| Method | Links | |
|---|---|---|
| Code-A1Code LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 85.21 | |
| Self-PlayCode LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 84.7 | |
| Golden TestsCode LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 84.68 | |
| /Code LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 83.69 | |
| Code-A1Code LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 83.52 | |
| Golden TestsCode LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 81.96 | |
| Self-PlayCode LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 81.86 | |
| /Code LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 77.63 | |
| Code-A1Code LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 72.69 | |
| Golden TestsCode LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 71.15 | |
| Self-PlayCode LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 70.64 | |
| /Code LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 63.42 |