Coding Performance (F1) on LiveCodeBench In-Distribution
47.4Initial F1 ScoreAutoPyVerifier
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AutoPyVerifierVerifier Model=GPT-5.42026.04 | 47.4 | 79.6 | 32.2 | |
| AutoPyVerifierVerifier Model=GPT-5.4 mini2026.04 | 43.7 | 56.8 | 13.1 | |
| AutoPyVerifierVerifier Model=Gemini-3.1 Pro2026.04 | 38.6 | 64 | 25.4 | |
| AutoPyVerifierVerifier Model=Gemini-3.1 Fl2026.04 | 37.8 | 49.9 | 12.1 | |
| AutoPyVerifierVerifier Model=Claude Hai-4.52026.04 | 37.8 | 37.8 | 0 |