Code Generation on LiveCodeBench lite v5 (test)
64.07AccuracyGemini-2.5-flash
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-2.5-flashInference Strategy=Zero-shot2026.01 | 64.07 | |
| PoTBase Model=Qwen3-4B-Instruct-25072026.01 | 57.49 | |
| Claude-Opus-4Inference Strategy=Zero-shot2026.01 | 50.3 | |
| RethinkMCTSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 49.83 | |
| LATSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 47.51 | |
| Best of N (N=20)Base Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 43.61 | |
| ToTBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 43.11 | |
| AB-MCTSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 39.52 | |
| PG-TDBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 38.12 | |
| Qwen3-235B-A22BInference Strategy=Zero-shot2026.01 | 35.33 | |
| CODETBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 34.43 | |
| ReflexionBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 34.13 | |
| Deepseek-V3Inference Strategy=Zero-shot2026.01 | 31.74 | |
| RAPBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 31.73 | |
| Few-shot + CoTBase Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 31.13 | |
| Zero-shotBase Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 30.54 | |
| Self-RefineBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 30.54 | |
| Gpt-4oInference Strategy=Zero-shot2026.01 | 29.94 |