Code Generation on BigCodeBench (avg@32)
52.46avg@32Code-A1
Evaluation results
| Method | Links | |
|---|---|---|
| Code-A1Code LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 52.46 | |
| Golden TestsCode LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 52.28 | |
| Self-PlayCode LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 52.25 | |
| /Code LLM=Qwen2.5-Coder-7B-Instruct2026.03 | 49.41 | |
| Code-A1Code LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 45.85 | |
| Golden TestsCode LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 45.41 | |
| Self-PlayCode LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 45.09 | |
| /Code LLM=Qwen2.5-Coder-3B-Instruct2026.03 | 41.78 | |
| Code-A1Code LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 34.82 | |
| Golden TestsCode LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 34.23 | |
| Self-PlayCode LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 33.47 | |
| /Code LLM=Qwen2.5-Coder-1.5B-Instruct2026.03 | 29.34 |