Code Generation on LiveCodeBench lite v6 (test)
56.57AccuracyGemini-2.5-flash
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-2.5-flashInference Strategy=Zero-shot2026.01 | 56.57 | |
| PoTBase Model=Qwen3-4B-Instruct-25072026.01 | 49.71 | |
| RethinkMCTSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 44.12 | |
| LATSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 43.43 | |
| Claude-Opus-4Inference Strategy=Zero-shot2026.01 | 40 | |
| ToTBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 38.86 | |
| ReflexionBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 38.12 | |
| Best of N (N=20)Base Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 37.71 | |
| Qwen3-235B-A22BInference Strategy=Zero-shot2026.01 | 37.21 | |
| AB-MCTSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 36 | |
| PG-TDBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 34.32 | |
| CODETBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 30.13 | |
| Gpt-4oInference Strategy=Zero-shot2026.01 | 29.71 | |
| Few-shot + CoTBase Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 28.57 | |
| Self-RefineBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 28 | |
| RAPBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 28 | |
| Zero-shotBase Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 27.43 | |
| Deepseek-V3Inference Strategy=Zero-shot2026.01 | 16 |