Code Generation on LiveCodeBench v6 (test)
85.63AccuracyATLAS-MM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ATLAS-MMBackbone=Claude Sonnet 4.62026.06 | 85.63 | — | — | |
| Self-Refine (no early stop)Backbone=Claude Sonnet 4.62026.06 | 82.29 | — | — | |
| ATLASBackbone=Claude Sonnet 4.62026.06 | 82.29 | — | — | |
| Self-RefineBackbone=Claude Sonnet 4.62026.06 | 80.57 | — | — | |
| o4-mini-HProtocol=Zero-shot2026.06 | 80.2 | — | — | |
| Budget ForcingBackbone=Claude Sonnet 4.62026.06 | 80 | — | — | |
| Reward-model rerankingBackbone=Claude Sonnet 4.6, Reward Model=Skywork-Reward-V22026.06 | 78.86 | — | — | |
| Pass@1Backbone=Claude Sonnet 4.62026.06 | 77.14 | — | — | |
| o3-HProtocol=Zero-shot2026.06 | 75.8 | — | — | |
| o4-mini-MProtocol=Zero-shot2026.06 | 74.2 | — | — | |
| Gemini-2.5-ProProtocol=Zero-shot2026.06 | 73.6 | — | — | |
| DeepSeek-R1Protocol=Zero-shot2026.06 | 73.1 | — | — | |
| PACEModel=DeepSeek-R1-Distill-Qwen-7B2026.02 | 36.8 | 8,110 | 30.8 | |
| VanillaModel=DeepSeek-R1-Distill-Qwen-7B2026.02 | 35 | 11,723 | — | |
| PACEModel=DeepSeek-R1-Distill-Qwen-1.5B2026.02 | 18.3 | 10,286 | 26.5 | |
| VanillaModel=DeepSeek-R1-Distill-Qwen-1.5B2026.02 | 17.2 | 13,991 | — |