Human-robot teaming for household object rearrangement on VirtualHome 2.0 (test)
80Success RateCaPE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CaPEApproach=CaPE, Backbone=Gemini 3.0 pro2026.02 | 80 | 22.5 | 8.8 | 2,711 | |
| CaPEApproach=CaPE, Backbone=Claude Opus 4.52026.02 | 70 | 20.9 | 6.8 | 2,306 | |
| CaPEApproach=CaPE, Backbone=GPT 5.22026.02 | 65 | 18.2 | 10.1 | 2,539 | |
| CaPEApproach=CaPE, Backbone=Molmo 8B2026.02 | 55 | 19 | 1.7 | 1,740 | |
| CaPEApproach=CaPE, Backbone=Qwen 2.5 7B, Training Protocol=Finetuned2026.02 | 48.3 | 16.5 | 1.7 | 2,448 | |
| CaPEApproach=CaPE, Backbone=Qwen 2.5 72B2026.02 | 46.6 | 18 | 1.9 | 2,304 | |
| CaPEApproach=CaPE, Backbone=Qwen 2.5 7B2026.02 | 41.6 | 17.1 | 1.1 | 982 | |
| Planner-onlyApproach=Planner-only2026.02 | 8.3 | 4.9 | 3.2 | 0 | |
| VLM WaypointsApproach=VLM Waypoints, Backbone=Gemini 3 pro2026.02 | 8.3 | 2.1 | 4.6 | 1,794 | |
| VLM WaypointsApproach=VLM Waypoints, Backbone=Qwen 2.5 72B2026.02 | 5 | 0.7 | 3.2 | 1,539 | |
| RoboPointApproach=RoboPoint2026.02 | 3.3 | 0.8 | 2.1 | 507 | |
| VLM WaypointsApproach=VLM Waypoints, Backbone=GPT 5.22026.02 | 3.3 | 1.4 | 2.3 | 1,673 | |
| VLM WaypointsApproach=VLM Waypoints, Backbone=Claude Opus 4.52026.02 | 1.6 | 0.4 | 12.4 | 1,790 | |
| VLM AgentsApproach=VLM Agents, Backbone=Gemini 3.0 pro2026.02 | 0 | 0 | 17 | 4,238 | |
| VLM AgentsApproach=VLM Agents, Backbone=Claude Opus 4.52026.02 | 0 | 0 | 13.7 | 2,378 | |
| VLM AgentsApproach=VLM Agents, Backbone=GPT 5.22026.02 | 0 | 0 | 2.2 | 2,195 | |
| VLM AgentsApproach=VLM Agents, Backbone=Molmo 8B2026.02 | 0 | 0 | 2.2 | 2,175 | |
| VLM AgentsApproach=VLM Agents, Backbone=Qwen 2.5 72B2026.02 | 0 | 0 | 3 | 2,307 | |
| VLM WaypointsApproach=VLM Waypoints, Backbone=Molmo 8B2026.02 | 0 | 0 | 6 | 1,827 |