Code Generation on CodeForces (pass@1)
53.79pass@1Claude-Sonnet-4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude-Sonnet-4.5Institution=Anthropic2026.03 | 53.79 | |
| Gemini-2.5-ProInstitution=Google2026.03 | 52.4 | |
| ReflexiCoder-8B (Multiple)Institution=Ours, setup=full iterative reasoning-reflection setup2026.03 | 37.68 | |
| ReflexiCoder-8B (Single)Institution=Ours, setup=single-attempt without system prompt2026.03 | 37.34 | |
| GPT-5.1Institution=OpenAI2026.03 | 34.7 | |
| GPT-4.1Institution=OpenAI2026.03 | 31.37 | |
| Qwen3-8BInstitution=Alibaba2026.03 | 23.7 | |
| DeepCoder-14B-PreviewInstitution=rLLM2026.03 | 14.24 | |
| DeepCoder-1.5B-PreviewInstitution=rLLM2026.03 | 9.89 | |
| Qwen2.5-Coder-7B-InstructInstitution=Alibaba2026.03 | 6.39 | |
| Seed-Coder-8B-InstructInstitution=ByteDance2026.03 | 4.09 | |
| DeepSeek-Coder-7B-InstructInstitution=DeepSeek2026.03 | 2.56 | |
| CodeGemma-7B-ITInstitution=Google2026.03 | 1.88 | |
| CodeLlama-7b-InstructInstitution=Meta2026.03 | 0.68 |