Planning on NL-PDDL-Bench Logistics
78Planner Success RateLlama 3.1-8B with symbolic feedback
Evaluation results
| Method | Links | |
|---|---|---|
| Llama 3.1-8B with symbolic feedbackPlanning Mode=Planner-backed formalization, Backbone=LoRA Llama 3.1-8B, External Planner=Fast Downward2026.06 | 78 | |
| DeepSeek-R1Planning Mode=Direct generation (LLM-only)2026.06 | 56.8 | |
| Claude 4.6Planning Mode=Direct generation (LLM-only)2026.06 | 56.1 | |
| GPT-5.2Planning Mode=Direct generation (LLM-only)2026.06 | 42.3 | |
| GLM-5Planning Mode=Direct generation (LLM-only)2026.06 | 23.2 | |
| Claude 3.5Planning Mode=Direct generation (LLM-only)2026.06 | 22.6 | |
| GPT-4oPlanning Mode=Direct generation (LLM-only)2026.06 | 22.4 |