Embodied Task Planning on VirtualHome
73.5Overall PerformanceTaskGround
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| TaskGroundModel=GPT-52026.05 | 73.5 | — | — | — | — | — | — | |
| ReCAPA2026.04 | 70.5 | 44.2 | 72.8 | 94.5 | 91.7 | 64.9 | 84.6 | |
| Claude-4-Sonnet2026.04 | 69.1 | 43.5 | 73.6 | 92.7 | 90.9 | 51.4 | 86.8 | |
| o1-preview2026.04 | 65.8 | 42.7 | 71.1 | 93.2 | 89.4 | 48 | 72.4 | |
| Gemini-Pro2026.04 | 65.3 | 37.9 | 73.1 | 91.1 | 87 | 34.1 | 91.9 | |
| Claude-3-Sonnet2026.04 | 64.9 | 33 | 72.8 | 92 | 89.1 | 48.9 | 80.5 | |
| TaskGroundModel=DeepSeek-V4-Flash2026.05 | 63 | — | — | — | — | — | — | |
| GPT-4o2026.04 | 60.8 | 36.5 | 61.6 | 91.1 | 87.6 | 46.7 | 68.2 | |
| Claude-3-Opus2026.04 | 59.9 | 31.4 | 66.2 | 89.9 | 86.7 | 48.8 | 61.8 | |
| o1-mini2026.04 | 57.9 | 31.2 | 65.9 | 84.6 | 79.3 | 41.5 | 69 | |
| TaskGroundModel=GPT-4.12026.05 | 57 | — | — | — | — | — | — | |
| TaskGroundModel=GPT-4o2026.05 | 49 | — | — | — | — | — | — | |
| TaskGroundModel=Gemini-2.5-Flash2026.05 | 48.5 | — | — | — | — | — | — | |
| TaskGroundModel=Qwen3.5-9B2026.05 | 47.5 | — | — | — | — | — | — | |
| NaiveModel=GPT-52026.05 | 45.5 | — | — | — | — | — | — | |
| TaskGroundModel=MiMo-V2-Flash2026.05 | 43.5 | — | — | — | — | — | — | |
| TaskGroundModel=Gemma-3-12B2026.05 | 37 | — | — | — | — | — | — | |
| NaiveModel=DeepSeek-V4-Flash2026.05 | 28.5 | — | — | — | — | — | — | |
| NaiveModel=GPT-4.12026.05 | 14 | — | — | — | — | — | — | |
| NaiveModel=GPT-4o2026.05 | 8 | — | — | — | — | — | — | |
| NaiveModel=MiMo-V2-Flash2026.05 | 5 | — | — | — | — | — | — | |
| NaiveModel=Gemini-2.5-Flash2026.05 | 0.5 | — | — | — | — | — | — | |
| NaiveModel=Gemma-3-12B2026.05 | 0 | — | — | — | — | — | — | |
| NaiveModel=Qwen3.5-9B2026.05 | 0 | — | — | — | — | — | — |