Goal-oriented task execution on FullHome (BEHAVIOR)
64.6Goal Success Rate (SR)TaskGround
Evaluation Results
| Method | Links | |
|---|---|---|
| TaskGroundModel=GPT-4.12026.05 | 64.6 | |
| TaskGroundModel=GPT-52026.05 | 64.6 | |
| TaskGroundModel=GPT-4o2026.05 | 52.3 | |
| TaskGroundModel=DeepSeek-V4-Flash2026.05 | 47.7 | |
| NaiveModel=GPT-52026.05 | 46.2 | |
| TaskGroundModel=MiMo-V2-Flash2026.05 | 43.1 | |
| TaskGroundModel=Qwen3.5-9B2026.05 | 43.1 | |
| TaskGroundModel=Gemini-2.5-Flash2026.05 | 38.5 | |
| NaiveModel=GPT-4o2026.05 | 32.3 | |
| NaiveModel=GPT-4.12026.05 | 32.3 | |
| TaskGroundModel=Gemma-3-12B2026.05 | 29.2 | |
| NaiveModel=DeepSeek-V4-Flash2026.05 | 27.7 | |
| NaiveModel=MiMo-V2-Flash2026.05 | 10.8 | |
| NaiveModel=Gemini-2.5-Flash2026.05 | 9.2 | |
| NaiveModel=Qwen3.5-9B2026.05 | 1.5 | |
| NaiveModel=Gemma-3-12B2026.05 | 0 |