Code Generation on BigCodeBench-I Hard
28.4ScoreGPT-o1
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-o1Backbone=GPT-o12025.09 | 28.4 | |
| DeepSeek-V2.5-238BBackbone=DeepSeek-V2.5-238B2025.09 | 27 | |
| CRITIQUE-CODERBackbone=Qwen3-8B, Thinking Mode=true, Training Strategy=CRL2025.09 | 27 | |
| Qwen3-8B-RLBackbone=Qwen3-8B, Thinking Mode=true, Training Strategy=Standard RL2025.09 | 24.3 | |
| Baseline (Qwen3-8B)Backbone=Qwen3-8B, Thinking Mode=true, Training Strategy=Baseline2025.09 | 23.6 | |
| Qwen3-4B-RLBackbone=Qwen3-4B, Thinking Mode=true, Training Strategy=Standard RL2025.09 | 23 | |
| CRITIQUE-CODERBackbone=Qwen3-4B, Thinking Mode=true, Training Strategy=CRL2025.09 | 23 | |
| DeepSeek-R1-Distill-14BBackbone=DeepSeek-R1-Distill-14B2025.09 | 20.9 | |
| Baseline (Qwen3-4B)Backbone=Qwen3-4B, Thinking Mode=true, Training Strategy=Baseline2025.09 | 20.9 | |
| AceCoder-7BBackbone=AceCoder-7B2025.09 | 19.6 | |
| DeepCoder-14BBackbone=DeepCoder-14B2025.09 | 18.2 |