Code Generation on CodeContests
89.09Pass@1Solvita
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SolvitaBackbone=DeepSeek V4 Pro2026.05 | 89.09 | — | — | |
| SolvitaBackbone=GPT-5.42026.05 | 82.42 | — | — | |
| Codex CLIBackbone=GPT-5.42026.05 | 81.82 | — | — | |
| SolvitaBackbone=Claude Opus 4.62026.05 | 80.61 | — | — | |
| Claude CodeBackbone=Claude Opus 4.62026.05 | 80 | — | — | |
| Codex CLIBackbone=DeepSeek V4 Pro2026.05 | 79.39 | — | — | |
| SolvitaBackbone=Grok2026.05 | 78.18 | — | — | |
| Claude CodeBackbone=DeepSeek V4 Pro2026.05 | 76.97 | — | — | |
| Codex CLIBackbone=Grok2026.05 | 73.33 | — | — | |
| Claude CodeBackbone=Grok2026.05 | 72.12 | — | — | |
| Claude CodeBackbone=GPT-5.42026.05 | 70.91 | — | — | |
| AlphaCodiumBackbone=DeepSeek V4 Pro2026.05 | 70.91 | — | — | |
| Codex CLIBackbone=Claude Opus 4.62026.05 | 70.3 | — | — | |
| SolvitaBackbone=Qwen3.62026.05 | 69.7 | — | — | |
| MapCoderBackbone=DeepSeek V4 Pro2026.05 | 66.67 | — | — | |
| AlphaCodiumBackbone=Claude Opus 4.62026.05 | 64.24 | — | — | |
| MapCoderBackbone=Grok2026.05 | 62.42 | — | — | |
| AlphaCodiumBackbone=GPT-5.42026.05 | 60.61 | — | — | |
| AlphaCodiumBackbone=Grok2026.05 | 60.61 | — | — | |
| MapCoderBackbone=Claude Opus 4.62026.05 | 60 | — | — | |
| Codex CLIBackbone=Qwen3.62026.05 | 60 | — | — | |
| PromptBridgeSource Model=GPT-4o, Target Model=o4-mini2025.12 | 58.79 | — | — | |
| Claude CodeBackbone=Qwen3.62026.05 | 58.79 | — | — | |
| GPT-5 OptimizerSource Model=GPT-4o, Target Model=o4-mini2025.12 | 57.7 | — | — | |
| MapCoderBackbone=GPT-5.42026.05 | 57.58 | — | — | |
| PromptBridgeSource Model=GPT-4o, Target Model=o32025.12 | 56.36 | — | — | |
| GEPASource Model=GPT-4o, Target Model=o4-mini2025.12 | 55.15 | — | — | |
| AlphaCodiumBackbone=Qwen3.62026.05 | 53.33 | — | — | |
| MapCoderBackbone=Qwen3.62026.05 | 50.91 | — | — | |
| Direct TransferSource Model=GPT-4o, Target Model=o32025.12 | 48.61 | — | — | |
| MIPROv2Source Model=GPT-4o, Target Model=o4-mini2025.12 | 48.28 | — | — | |
| Single-passBackbone=DeepSeek V4 Pro2026.05 | 47.27 | — | — | |
| GEPASource Model=GPT-4o, Target Model=o32025.12 | 46.19 | — | — | |
| MIPROv2Source Model=GPT-4o, Target Model=o32025.12 | 45.65 | — | — | |
| Single-passBackbone=Claude Opus 4.62026.05 | 44.85 | — | — | |
| Direct TransferSource Model=GPT-4o, Target Model=o4-mini2025.12 | 40.12 | — | — | |
| ThinkCoderBackbone=GPT-4o2024.12 | 40.1 | — | — | |
| Single-passBackbone=GPT-5.42026.05 | 40 | — | — | |
| Single-passBackbone=Grok2026.05 | 38.18 | — | — | |
| LPWBackbone=GPT-4o2024.12 | 34.7 | — | — | |
| Single-passBackbone=Qwen3.62026.05 | 33.94 | — | — | |
| GPT-5 OptimizerSource Model=GPT-4o, Target Model=o32025.12 | 31.88 | — | — | |
| LDBBackbone=GPT-4o2024.12 | 29.3 | — | — | |
| PromptBridgeSource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.12 | 23.84 | — | — | |
| MIPROv2Source Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.12 | 21.82 | — | — | |
| Direct TransferSource Model=GPT-4o, Target Model=Gemma3-27B-it2025.12 | 21.21 | — | — | |
| Direct TransferSource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.12 | 20.2 | — | — | |
| GPT-5 OptimizerSource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.12 | 19.8 | — | — | |
| GPT-5 OptimizerSource Model=GPT-4o, Target Model=Qwen3-32B2025.12 | 19.8 | — | — | |
| GEPASource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.12 | 18.99 | — | — | |
| GPT-5 OptimizerSource Model=GPT-4o, Target Model=Gemma3-27B-it2025.12 | 18.99 | — | — | |
| PromptBridgeSource Model=GPT-4o, Target Model=Qwen3-32B2025.12 | 18.79 | — | — | |
| PromptBridgeSource Model=GPT-4o, Target Model=Gemma3-27B-it2025.12 | 16.97 | — | — | |
| ARCHCODECoT=true, Code Filtering=true, NFRs=true2024.08 | 16.52 | 16.67 | 17.37 | |
| MPSCCoT=true, Code Filtering=true, NFRs=false, Base Model=GPT-3.5-Turbo2024.08 | 14.39 | 17.16 | 17.76 | |
| Direct TransferSource Model=GPT-4o, Target Model=Qwen3-32B2025.12 | 13.94 | — | — | |
| ALGOCoT=true, Code Filtering=false, NFRs=false, Base Model=GPT-3.5-Turbo2024.08 | 12 | 12 | — | |
| CODETCoT=true, Code Filtering=true, NFRs=false, Base Model=GPT-3.5-Turbo2024.08 | 9.92 | 10.18 | 10.3 | |
| MBR-EXECCoT=true, Code Filtering=true, NFRs=false, Base Model=GPT-3.5-Turbo2024.08 | 8.25 | 8.87 | 9.08 | |
| PromptBridgeSource Model=GPT-4o, Target Model=Llama3.1-8B-Instruct2025.12 | 7.68 | — | — | |
| Direct TransferSource Model=GPT-4o, Target Model=Llama3.1-8B-Instruct2025.12 | 7.47 | — | — | |
| GPT-5 OptimizerSource Model=GPT-4o, Target Model=Llama3.1-8B-Instruct2025.12 | 7.27 | — | — | |
| BRAINSTORMCoT=true, Code Filtering=false, NFRs=false, Base Model=GPT-3.5-Turbo2024.08 | 7 | — | 14.7 | |
| GPT-4CoT=false, Code Filtering=false, NFRs=false2024.08 | 6.07 | 8.23 | 11.67 | |
| GPT-3.5-Turbo + CoTCoT=true, Code Filtering=false, NFRs=false2024.08 | 5.82 | 8.57 | 13.53 | |
| GPT-4oModel Role=Source Model2025.12 | 5.45 | — | — | |
| GPT-4oSource Model=GPT-4o, Evaluation Context=Source Performance Reference2025.12 | 5.45 | — | — | |
| GPT-3.5-TurboCoT=false, Code Filtering=false, NFRs=false2024.08 | 4.79 | 7.02 | 10.06 |