Planning on PlanBench-BW NL
42.7Accuracy (%)Qwen3-Base SAT
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3-Base SATSize=3×4B2026.04 | 42.7 | |
| OpenAI o3-mini (medium)Size=/2026.04 | 41.2 | |
| Qwen3 (thinking)Size=32B2026.04 | 40.3 | |
| Qwen3-A3BSize=30B2026.04 | 39.1 | |
| DeepSeek-R1 Distill LlamaSize=70B2026.04 | 39.1 | |
| Qwen3-Base Debate + JudgeSize=3×8B2026.04 | 39.1 | |
| Qwen3-Base Role-playSize=3×8B2026.04 | 38.7 | |
| Qwen3-Base DebateSize=3×8B2026.04 | 38.3 | |
| QwQSize=32B2026.04 | 37.9 | |
| Llama 3.1-Base SATSize=3×8B2026.04 | 37.1 | |
| GPT-4o-mini-2024-07-18Size=/2026.04 | 34.7 | |
| Qwen3-BaseSize=32B2026.04 | 34.3 | |
| Qwen2.5-BaseSize=72B2026.04 | 33.7 | |
| Qwen3-Base-DAPOSize=8B2026.04 | 33.5 | |
| Qwen3-Base-GRPOSize=8B2026.04 | 33.1 | |
| Qwen3-BaseSize=14B2026.04 | 29.7 |