Code Generation on OJBench ICPC 2025 (test)
19.18AccuracyPoT
Evaluation Results
| Method | Links | |
|---|---|---|
| PoTBase Model=Qwen3-4B-Instruct-25072026.01 | 19.18 | |
| ToTBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 16.43 | |
| AB-MCTSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 15.07 | |
| RethinkMCTSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 15.07 | |
| LATSBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 13.69 | |
| ReflexionBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 10.59 | |
| PG-TDBase Model=Qwen3-4B-Instruct-2507, Method Category=Search-Based Reasoning2026.01 | 10.59 | |
| Deepseek-V3Inference Strategy=Zero-shot2026.01 | 9.59 | |
| Best of N (N=20)Base Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 9.59 | |
| Gemini-2.5-flashInference Strategy=Zero-shot2026.01 | 8.22 | |
| Few-shot + CoTBase Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 8.22 | |
| CODETBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 8.22 | |
| RAPBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 8.22 | |
| Gpt-4oInference Strategy=Zero-shot2026.01 | 6.8 | |
| Claude-Opus-4Inference Strategy=Zero-shot2026.01 | 6.8 | |
| Zero-shotBase Model=Qwen3-4B-Instruct-2507, Method Category=Standard Inference2026.01 | 6.8 | |
| Qwen3-235B-A22BInference Strategy=Zero-shot2026.01 | 5.48 | |
| Self-RefineBase Model=Qwen3-4B-Instruct-2507, Method Category=Self-Refinement / Debugging2026.01 | 5.48 |