Competitive Programming Agent Evaluation on ALE Bench
1,909.4Final PerformanceLeeroo
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LeerooModel=Gemini-2.5-pro2026.01 | 1,909.4 | 6.1 | 914.8 | |
| ALE-AgentModel=Gemini-2.5-pro2026.01 | 1,879.3 | 6.8 | 1,003.3 | |
| ALE SequentialModel=Gemini-2.5-pro2026.01 | 1,198 | 54.1 | 111 | |
| RiVERBackbone=Qwen3-8B2026.06 | 987 | 77.5 | — | |
| RiVERBackbone=GLM-Z1-9B-04142026.06 | 962 | 78.8 | — | |
| Rank-uniformBackbone=Qwen3-8B2026.06 | 960 | 78.9 | — | |
| Instance-NormBackbone=Qwen3-8B2026.06 | 935 | 80.1 | — | |
| Rank-uniformBackbone=GLM-Z1-9B-04142026.06 | 931 | 80.2 | — | |
| Instance-NormBackbone=GLM-Z1-9B-04142026.06 | 929 | 80.6 | — | |
| Raw-BinaryBackbone=Qwen3-8B2026.06 | 926 | 80.9 | — | |
| RS-GRPOBackbone=GLM-Z1-9B-04142026.06 | 916 | 81.6 | — | |
| Raw-BinaryBackbone=GLM-Z1-9B-04142026.06 | 915 | 81.8 | — | |
| RS-GRPOBackbone=Qwen3-8B2026.06 | 904 | 82.5 | — | |
| Raw-GRPOBackbone=Qwen3-8B2026.06 | 903 | 82.5 | — | |
| Raw-GRPOBackbone=GLM-Z1-9B-04142026.06 | 886 | 83.6 | — | |
| Qwen3-8BBackbone=Qwen3-8B2026.06 | 845 | 86.4 | — | |
| ALE one-shotModel=Gemini-2.5-pro2026.01 | 832 | 88.4 | 4.7 | |
| GLM-Z1-9B-0414Backbone=GLM-Z1-9B-04142026.06 | 805 | 88.2 | — |