Planning on PlanBench Mystery Blocksworld
87.3Success RateLlama 3.1-8B with symbolic feedback
Evaluation results
| Method | Links | |
|---|---|---|
| Llama 3.1-8B with symbolic feedbackPlanning Mode=Planner-backed formalization, Backbone=LoRA Llama 3.1-8B, External Planner=Fast Downward2026.06 | 87.3 | |
| GLM-5Planning Mode=Direct generation (LLM-only)2026.06 | 82 | |
| Claude 4.6Planning Mode=Direct generation (LLM-only)2026.06 | 60.5 | |
| DeepSeek-R1Planning Mode=Direct generation (LLM-only)2026.06 | 43.3 | |
| GPT-5.2Planning Mode=Direct generation (LLM-only)2026.06 | 6.2 | |
| GPT-4oPlanning Mode=Direct generation (LLM-only)2026.06 | 0 | |
| Claude 3.5Planning Mode=Direct generation (LLM-only)2026.06 | 0 |