Planning on BlocksWorld (Valid %, Successful %)
100Success RateGPT5
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT52025.08 | 100 | — | |
| LLM4PredictLLM Component=DeepSeek-R12025.08 | 98 | — | |
| Claude-sonnet-4-202505142025.08 | 92 | — | |
| Decomposition2025.08 | 80 | — | |
| LLM4InspireLLM Component=DeepSeek-R12025.08 | 74 | — | |
| DeepSeek-R12025.08 | 70 | — | |
| L-ICLNumber of training examples (m)=60, Input representation (ASCII grid)=true2026.01 | 66 | 68 | |
| L-ICLNumber of training examples (m)=60, Input representation (ASCII grid)=false2026.01 | 66 | 68 | |
| Fast DownwardVersion=24.06.12025.08 | 52 | — | |
| ReActInference feedback=Oracle feedback, Input representation (ASCII grid)=true2026.01 | 51 | 51 | |
| Self-RefineReasoning samples (k)=5, Input representation (ASCII grid)=true2026.01 | 49 | 49 | |
| L-ICLNumber of training examples (m)=0, Input representation (ASCII grid)=true2026.01 | 48 | 50 | |
| L-ICLNumber of training examples (m)=0, Input representation (ASCII grid)=false2026.01 | 48 | 50 | |
| ReActPrompting strategy=ReAct (Prompt-Only), Input representation (ASCII grid)=true2026.01 | 45 | 46 | |
| ToTPrompting strategy=ToT (Prompt-Only), Input representation (ASCII grid)=true2026.01 | 40 | 50 | |
| RAG-ICLContext size (characters)=20k, Input representation (ASCII grid)=true2026.01 | 32 | 32 | |
| Self-ConsistencyReasoning samples (k)=5, Input representation (ASCII grid)=true2026.01 | 31 | 31 | |
| RAG-ICLContext size (characters)=10k, Input representation (ASCII grid)=true2026.01 | 25 | 25 | |
| Zero-ShotInput representation (ASCII grid)=true2026.01 | 10 | 10 | |
| RAP2025.08 | 2 | — |