Proactive Assistance on VirtualHome v2.2.4 (Apartments #3 and #6)
38.41Speedup (%)Gemini-3-Flash
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini-3-FlashTask ID=32026.05 | 38.41 | 50 | |
| MindZeroTask ID=3, Base Model=Qwen3-4B2026.05 | 37.5 | 51 | |
| Gemini-3-FlashTask ID=132026.05 | 26.1 | 91 | |
| Qwen3-4BTask ID=32026.05 | 23.67 | 56 | |
| Gemini-3-FlashTask ID=Average2026.05 | 23.4 | — | |
| Gemini-3-FlashTask ID=52026.05 | 21.55 | 39 | |
| MindZeroTask ID=Average, Base Model=Qwen3-4B2026.05 | 19.7 | — | |
| MindZeroTask ID=13, Base Model=Qwen3-4B2026.05 | 18.28 | 97 | |
| MindZeroTask ID=8, Base Model=Qwen3-4B2026.05 | 15.45 | 41 | |
| Qwen3-4BTask ID=82026.05 | 9.23 | 43 | |
| Qwen3-4BTask ID=Standard Error2026.05 | 9 | — | |
| MindZeroTask ID=5, Base Model=Qwen3-4B2026.05 | 7.63 | 44 | |
| Gemini-3-FlashTask ID=82026.05 | 7.58 | 44 | |
| Gemini-3-FlashTask ID=Standard Error2026.05 | 6.4 | — | |
| MindZeroTask ID=Standard Error, Base Model=Qwen3-4B2026.05 | 6.3 | — | |
| Qwen3-4BTask ID=Average2026.05 | 2.62 | — | |
| Qwen3-4BTask ID=132026.05 | -3.92 | 119 | |
| Qwen3-4BTask ID=52026.05 | -18.5 | 58 | |
| Single HumanTask ID=32026.05 | — | 70 | |
| Single HumanTask ID=52026.05 | — | 47 | |
| Single HumanTask ID=82026.05 | — | 47 | |
| Single HumanTask ID=132026.05 | — | 114 |