Planning on NL-PDDL-Bench Zenotravel
99Success RateLlama 3.1-8B with symbolic feedback
Evaluation Results
| Method | Links | |
|---|---|---|
| Llama 3.1-8B with symbolic feedbackPlanning Mode=Planner-backed formalization, Backbone=LoRA Llama 3.1-8B, External Planner=Fast Downward2026.06 | 99 | |
| DeepSeek-R1Planning Mode=Direct generation (LLM-only)2026.06 | 98.9 | |
| Claude 4.6Planning Mode=Direct generation (LLM-only)2026.06 | 97.9 | |
| GLM-5Planning Mode=Direct generation (LLM-only)2026.06 | 75.8 | |
| GPT-5.2Planning Mode=Direct generation (LLM-only)2026.06 | 64.2 | |
| Claude 3.5Planning Mode=Direct generation (LLM-only)2026.06 | 57.9 | |
| GPT-4oPlanning Mode=Direct generation (LLM-only)2026.06 | 33.7 |