Code Generation on LiveCodeBench Hard
63.9Pass@1DreamPRM-Code
Evaluation Results
| Method | Links | |
|---|---|---|
| DreamPRM-CodeEvaluation Protocol=Test-Time Scaling, Base Model=O4-mini-high2025.12 | 63.9 | |
| ORMEvaluation Protocol=Test-Time Scaling, Base Model=O4-mini-high2025.12 | 62.3 | |
| HEARTModel=Gemini 2.5 Flash2025.09 | 60.76 | |
| HEARTModel=Gemini 2.5 Pro2025.09 | 60.22 | |
| O3Evaluation Protocol=Leaderboard2025.12 | 57.4 | |
| O4-mini-highEvaluation Protocol=Leaderboard2025.12 | 57.4 | |
| Self ReflectionModel=Gemini 2.5 Flash2025.09 | 56.96 | |
| WaitModel=Gemini 2.5 Flash2025.09 | 56.96 | |
| CoTModel=Gemini 2.5 Flash2025.09 | 56.65 | |
| CoTModel=Gemini 2.5 Pro2025.09 | 55.06 | |
| Self ReflectionModel=Gemini 2.5 Pro2025.09 | 54.54 | |
| WaitModel=Gemini 2.5 Pro2025.09 | 54.11 | |
| Gemini-2.5Evaluation Protocol=Leaderboard2025.12 | 52.5 | |
| DeepSeek-R1Evaluation Protocol=Leaderboard2025.12 | 47.2 | |
| VanillaModel=Gemini 2.5 Pro2025.09 | 45.73 | |
| VanillaModel=Gemini 2.5 Flash2025.09 | 31 | |
| DEPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | 9.7 | |
| XPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | 9.4 | |
| POPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | 9.2 | |
| DPO-iter3Base Model=Qwen2.5-Coder-7B-Instruct, Iteration=32026.05 | 8.7 | |
| Qwen2.5-Coder-7B-Instruct2026.05 | 6.9 | |
| VanillaModel=Gemma3-12b-Instruct2025.09 | 4.29 | |
| Self ReflectionModel=Gemma3-12b-Instruct2025.09 | 3.71 | |
| WaitModel=Gemma3-12b-Instruct2025.09 | 3.71 | |
| HEARTModel=Gemma3-12b-Instruct2025.09 | 3.71 | |
| CoTModel=Gemma3-12b-Instruct2025.09 | 3.43 |