Process-constrained task execution on FullHome (VirtualHome)
51.4Process Success RateTaskGround
Evaluation Results
| Method | Links | |
|---|---|---|
| TaskGroundModel=GPT-52026.05 | 51.4 | |
| TaskGroundModel=GPT-52026.05 | 46 | |
| TaskGroundModel=GPT-4.12026.05 | 45.7 | |
| TaskGroundModel=DeepSeek-V4-Flash2026.05 | 42.9 | |
| TaskGroundModel=MiMo-V2-Flash2026.05 | 40 | |
| TaskGroundModel=Qwen3.5-9B2026.05 | 40 | |
| TaskGroundModel=Gemini-2.5-Flash2026.05 | 37.1 | |
| TaskGroundModel=GPT-4o2026.05 | 37 | |
| TaskGroundModel=DeepSeek-V4-Flash2026.05 | 36 | |
| TaskGroundModel=GPT-4o2026.05 | 35.5 | |
| TaskGroundModel=Gemma-3-12B2026.05 | 34.3 | |
| TaskGroundModel=GPT-4.12026.05 | 34 | |
| NaiveModel=GPT-52026.05 | 31.4 | |
| TaskGroundModel=Qwen3.5-9B2026.05 | 29 | |
| NaiveModel=GPT-52026.05 | 26 | |
| NaiveModel=DeepSeek-V4-Flash2026.05 | 25.7 | |
| NaiveModel=GPT-4o2026.05 | 23.3 | |
| TaskGroundModel=MiMo-V2-Flash2026.05 | 23 | |
| TaskGroundModel=Gemini-2.5-Flash2026.05 | 19 | |
| TaskGroundModel=Gemma-3-12B2026.05 | 19 | |
| NaiveModel=GPT-4.12026.05 | 17.1 | |
| NaiveModel=DeepSeek-V4-Flash2026.05 | 15 | |
| NaiveModel=Qwen3.5-9B2026.05 | 14.3 | |
| NaiveModel=MiMo-V2-Flash2026.05 | 11.4 | |
| NaiveModel=Qwen3.5-9B2026.05 | 10 | |
| NaiveModel=Gemini-2.5-Flash2026.05 | 5.7 | |
| NaiveModel=Gemma-3-12B2026.05 | 5.7 | |
| NaiveModel=GPT-4o2026.05 | 5 | |
| NaiveModel=GPT-4.12026.05 | 4 | |
| NaiveModel=Gemini-2.5-Flash2026.05 | 2 | |
| NaiveModel=MiMo-V2-Flash2026.05 | 1 | |
| NaiveModel=Gemma-3-12B2026.05 | 1 |