Coding on LiveCodeBench Out-of-Distribution
47.7Initial F1AutoPyVerifier
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AutoPyVerifierVerifier Model=GPT-5.4 mini2026.04 | 47.7 | 53.9 | 6.2 | |
| AutoPyVerifierVerifier Model=GPT-5.42026.04 | 43.9 | 78.7 | 34.8 | |
| AutoPyVerifierVerifier Model=Gemini-3.1 Pro2026.04 | 41.2 | 58.8 | 17.6 | |
| AutoPyVerifierVerifier Model=Gemini-3.1 Fl2026.04 | 41.2 | 58.8 | 17.6 | |
| AutoPyVerifierVerifier Model=Claude Hai-4.52026.04 | 41.2 | 41.2 | 0 |