Long-horizon Task Execution on Minecraft Long-horizon Tasks
100WoodHuman
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Human2025.02 | 100 | 100 | 86 | 17 | 16 | 33 | 28 | — | |
| Optimus-2Policy=GOAP2025.02 | 99 | 93 | 53 | 9 | 13 | 28 | 21 | — | |
| Optimus-2Dependency=Oracle, Planner LLM=GPT-4V2025.05 | 99 | 93 | 53 | 9 | 13 | 28 | 21 | 45 | |
| Optimus-1Policy=STEVE-12025.02 | 98 | 92 | 46 | 8 | 11 | 25 | 19 | — | |
| Optimus-1Dependency=Oracle, Planner LLM=GPT-4V2025.05 | 98 | 92 | 46 | 8 | 11 | 25 | 19 | 43 | |
| XENON *Dependency=Oracle, Planner LLM=Qwen2.5-VL-7B2025.05 | 95 | 93 | 83 | 73 | 75 | 75 | 61 | 79 | |
| Jarvis-1Policy=STEVE-12025.02 | 93 | 89 | 36 | 7 | 8 | 16 | 15 | — | |
| Jarvis-1Dependency=Oracle, Planner LLM=GPT-42025.05 | 93 | 89 | 36 | 7 | 8 | 16 | 15 | 38 | |
| Optimus-1†Dependency=Oracle, Planner LLM=Qwen2.5-VL-7B2025.05 | 92 | 80 | 22 | 9 | 10 | 4 | 17 | 34 | |
| XENONDependency=Learned, Planner LLM=Qwen2.5-VL-7B2025.05 | 85 | 81 | 46 | 74 | 64 | 0 | 28 | 54 | |
| DEPSPolicy=STEVE-12025.02 | 77 | 48 | 16 | 0 | 1 | 0 | 10 | — | |
| DEPSDependency=None, Planner LLM=Codex2025.05 | 77 | 48 | 16 | 0 | 1 | 0 | 10 | 22 | |
| Pure GPT-4VPolicy=GOAP, Context=zero-shot manner to generate executable sub-goals2025.02 | 50 | 31 | 12 | 2 | 1 | 3 | 3 | — | |
| Pure GPT-4VPolicy=STEVE-1, Context=zero-shot manner to generate executable sub-goals2025.02 | 41 | 20 | 0 | 0 | 0 | 0 | 0 | — | |
| Pure GPT-4VPolicy=VPT (text), Context=zero-shot manner to generate executable sub-goals2025.02 | 22 | 8 | 0 | 0 | 0 | 0 | 0 | — |