Competitive Programming on ICPC-Eval (Refine@5, #T)
48.3Refine@5Gemini-3-pro
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gemini-3-proBackbone=Gemini-3-pro2026.05 | 48.3 | 1.4 | |
| o4-mini-highBackbone=o4-mini-high2026.05 | 42.4 | 1.2 | |
| DeepSeek-V3.2-ReasonerBackbone=DeepSeek-V3.2-Reasoner2026.05 | 33.9 | 1.6 | |
| CP-AgentBackbone=DeepSeek-V3.2-Chat2026.05 | 33.9 | 1.3 | |
| ReflexionBackbone=DeepSeek-V3.2-Chat2026.05 | 32.5 | 1.2 | |
| CodeSimBackbone=DeepSeek-V3.2-Chat2026.05 | 29.7 | 1.2 | |
| AlphaCodiumBackbone=DeepSeek-V3.2-Chat2026.05 | 29.7 | 1.7 | |
| ReflexionBackbone=Qwen3-235B-A22B-Instruct2026.05 | 27.1 | 1.2 | |
| CP-AgentBackbone=Qwen3-235B-A22B-Instruct2026.05 | 26.3 | 1.6 | |
| CodeSimBackbone=Qwen3-235B-A22B-Instruct2026.05 | 25.4 | 2.1 | |
| AlphaCodiumBackbone=Qwen3-235B-A22B-Instruct2026.05 | 25.4 | 1.6 | |
| DeepSeek-V3.2-ChatBackbone=DeepSeek-V3.2-Chat2026.05 | 22.9 | 1.2 | |
| Qwen3-235B-A22B-InstructBackbone=Qwen3-235B-A22B-Instruct2026.05 | 21.2 | 1.4 | |
| DeepSeek-R1Backbone=DeepSeek-R12026.05 | 14.4 | 2.1 | |
| ReflexionBackbone=GPT-4o2026.05 | 7.7 | 2.4 | |
| CP-AgentBackbone=GPT-4o2026.05 | 7.7 | 1.6 | |
| AlphaCodiumBackbone=GPT-4o2026.05 | 5.9 | 2.7 | |
| CodeSimBackbone=GPT-4o2026.05 | 5.1 | 1.5 | |
| GPT-4oBackbone=GPT-4o2026.05 | 3.4 | 1.8 |