Code Generation on MBPP+ (avg@32)
74.5Average Success Rate (@32)Code-A1
Evaluation Results
| Method | Links | |
|---|---|---|
| Code-A1Code LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 74.5 | |
| Self-PlayCode LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 74.23 | |
| Golden TestsCode LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 74.16 | |
| /Code LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 71.95 | |
| Code-A1Code LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 69.07 | |
| Golden TestsCode LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 68.05 | |
| Self-PlayCode LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 67.06 | |
| Self-PlayCode LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 63.54 | |
| Code-A1Code LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 63.33 | |
| Golden TestsCode LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 63.3 | |
| /Code LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 63.12 | |
| /Code LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 60.87 |