Code Generation on LiveCodeBench Medium
96.79AccuracyWait
Evaluation results
| Method | Links | ||
|---|---|---|---|
| WaitModel=Gemini 3 Pro, Strategy=Wait2025.09 | 96.79 | — | |
| HEARTModel=Gemini 3 Pro, Strategy=HEART2025.09 | 96.78 | — | |
| HEARTModel=Gemini 3 Flash, Strategy=HEART2025.09 | 96.53 | — | |
| Self ReflectionModel=Gemini 3 Pro, Strategy=Self Reflection2025.09 | 96.53 | — | |
| CoTModel=Gemini 3 Pro, Strategy=CoT2025.09 | 96.52 | — | |
| Self ReflectionModel=Gemini 3 Flash, Strategy=Self Reflection2025.09 | 96 | — | |
| CoTModel=Gemini 3 Flash, Strategy=CoT2025.09 | 95.73 | — | |
| WaitModel=Gemini 3 Flash, Strategy=Wait2025.09 | 95.73 | — | |
| HEARTModel=Gemini 2.5 Pro2025.09 | 84.57 | — | |
| HEARTModel=Gemini 2.5 Flash2025.09 | 84.37 | — | |
| WaitModel=Gemini 2.5 Pro2025.09 | 83.02 | — | |
| CoTModel=Gemini 2.5 Pro2025.09 | 82.47 | — | |
| VanillaModel=Gemini 2.5 Flash2025.09 | 82.25 | — | |
| Self ReflectionModel=Gemini 2.5 Pro2025.09 | 82.14 | — | |
| Self ReflectionModel=Gemini 2.5 Flash2025.09 | 81.35 | — | |
| VanillaModel=Gemini 2.5 Pro2025.09 | 81.2 | — | |
| WaitModel=Gemini 2.5 Flash2025.09 | 80.17 | — | |
| CoTModel=Gemini 2.5 Flash2025.09 | 80.02 | — | |
| HEARTModel=Gemma3-12b-Instruct2025.09 | 13.32 | — | |
| Self ReflectionModel=Gemma3-12b-Instruct2025.09 | 13.05 | — | |
| CoTModel=Gemma3-12b-Instruct2025.09 | 13.05 | — | |
| WaitModel=Gemma3-12b-Instruct2025.09 | 13.05 | — | |
| VanillaModel=Gemma3-12b-Instruct2025.09 | 12.53 | — | |
| DeepSeek-R1Evaluation Protocol=Leaderboard2025.12 | — | 77.7 | |
| DEPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | — | 18.8 | |
| DPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | — | 14.5 | |
| DreamPRM-CodeEvaluation Protocol=Test-Time Scaling, Base Model=O4-mini-high2025.12 | — | 92.3 | |
| Gemini-2.5Evaluation Protocol=Leaderboard2025.12 | — | 82.1 | |
| O3Evaluation Protocol=Leaderboard2025.12 | — | 71.8 | |
| O4-mini-highEvaluation Protocol=Leaderboard2025.12 | — | 89.7 | |
| ORMEvaluation Protocol=Test-Time Scaling, Base Model=O4-mini-high2025.12 | — | 89.7 | |
| POPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | — | 15 | |
| Qwen2.5-Coder-7B-Instruct2026.05 | — | 3.8 | |
| XPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | — | 15.2 |