Robotic Planning on LEMMA Single-Agent Underspecified
99Success Rate (SR)SG-CoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SG-CoTLLM=Gemini-2.5-Flash2026.03 | 99 | 98 | |
| InnerMonoLLM=Gemini-2.5-Flash2026.03 | 95 | 93 | |
| SG-CoTLLM=Gemini-2.5-Flash, Scene Graph Edges=false2026.03 | 87 | 54 | |
| SG-CoTLLM=Gemini-2.5-Flash, Iterations=12026.03 | 82 | 75 | |
| ProgPromptLLM=Gemini-2.5-Flash2026.03 | 79 | 77 | |
| SG-CoTLLM=Qwen3-VL-2B2026.03 | 77 | 67 | |
| InnerMonoLLM=Qwen3-VL-2B2026.03 | 75 | 62 | |
| SG-CoTLLM=Qwen3-VL-2B, Scene Graph Edges=false2026.03 | 75 | 52 | |
| CLARALLM=Gemini-2.5-Flash2026.03 | 70 | 60 | |
| SG-CoTLLM=Qwen3-VL-2B, Iterations=12026.03 | 69 | 62 | |
| SG-CoTLLM=Gemini-2.5-Flash, Iterations=02026.03 | 68 | 52 | |
| CLARALLM=Qwen3-VL-2B2026.03 | 63 | 52 | |
| ProgPromptLLM=Qwen3-VL-2B2026.03 | 58 | 47 | |
| SG-CoTLLM=Qwen3-VL-2B, Iterations=02026.03 | 44 | 28 |