Code Generation on BigCodeBench-I Full
50.4ScoreGPT-o1
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-o1Backbone=GPT-o12025.09 | 50.4 | |
| DeepSeek-V2.5-238BBackbone=DeepSeek-V2.5-238B2025.09 | 48.9 | |
| CRITIQUE-CODERBackbone=Qwen3-8B, Thinking Mode=true, Training Strategy=CRL2025.09 | 46.6 | |
| Baseline (Qwen3-8B)Backbone=Qwen3-8B, Thinking Mode=true, Training Strategy=Baseline2025.09 | 44.6 | |
| Qwen3-8B-RLBackbone=Qwen3-8B, Thinking Mode=true, Training Strategy=Standard RL2025.09 | 44.5 | |
| AceCoder-7BBackbone=AceCoder-7B2025.09 | 43.3 | |
| CRITIQUE-CODERBackbone=Qwen3-4B, Thinking Mode=true, Training Strategy=CRL2025.09 | 43.1 | |
| Baseline (Qwen3-4B)Backbone=Qwen3-4B, Thinking Mode=true, Training Strategy=Baseline2025.09 | 42 | |
| Qwen3-4B-RLBackbone=Qwen3-4B, Thinking Mode=true, Training Strategy=Standard RL2025.09 | 40.6 | |
| DeepCoder-14BBackbone=DeepCoder-14B2025.09 | 38.2 | |
| DeepSeek-R1-Distill-14BBackbone=DeepSeek-R1-Distill-14B2025.09 | 38.1 |