Simulation Task Planning on BEHAVIOR-1K 15 tasks
100BT ValidGPT-5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5Prompting=CoT2026.03 | 100 | 100 | 100 | |
| GPT-5Prompting=ZS2026.03 | 100 | 100 | 100 | |
| Gemma-3 4BVar.=FT, Prompting=CoT2026.03 | 100 | 87 | 93 | |
| Gemma-3 4BVar.=FT, Prompting=ZS2026.03 | 100 | 73 | 80 | |
| Qwen2.5-VLVar.=FT, Prompting=CoT2026.03 | 100 | 67 | 87 | |
| Qwen2.5-VLVar.=FT, Prompting=ZS2026.03 | 100 | 47 | 67 | |
| SmolVLM2Var.=FT, Prompting=ZS2026.03 | 27 | 0 | 7 | |
| SmolVLM2Var.=FT, Prompting=CoT2026.03 | 7 | 0 | 0 | |
| Gemma-3 4BVar.=Base, Prompting=CoT2026.03 | 0 | 0 | 0 | |
| Gemma-3 4BVar.=Base, Prompting=ZS2026.03 | 0 | 0 | 0 | |
| Qwen2.5-VLVar.=Base, Prompting=CoT2026.03 | 0 | 0 | 0 | |
| Qwen2.5-VLVar.=Base, Prompting=ZS2026.03 | 0 | 0 | 0 | |
| SmolVLM2Var.=Base, Prompting=CoT2026.03 | 0 | 0 | 0 | |
| SmolVLM2Var.=Base, Prompting=ZS2026.03 | 0 | 0 | 0 |