Sequential Household Interaction Success on ALFWorld OOD (test)
96Pick Success RateReflexion
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| ReflexionSupervision=Access to text env, Agent type=LLM2026.05 | 96 | 94 | 100 | 81 | 83 | 88 | 91 | |
| DEPSSupervision=Access to text env, Agent type=LLM2026.05 | 93 | 100 | 50 | 80 | 100 | 0 | 76 | |
| VIPER BCSupervision=No access to text env, Agent type=VLM+LLM2026.05 | 80 | 77 | 67 | 87 | 71 | 53 | 72 | |
| VIPER BC+RLSupervision=No access to text env, Agent type=VLM+LLM2026.05 | 80 | 77 | 77 | 92 | 71 | 53 | 75 | |
| PRISM BCSupervision=No access to text env, Agent type=VLM+LLM2026.05 | 80 | 77 | 81 | 87 | 71 | 62 | 77 | |
| PRISM BC+RLSupervision=No access to text env, Agent type=VLM+LLM2026.05 | 80 | 83 | 84 | 92 | 71 | 67 | 80 | |
| EMAC+Supervision=Access to text env, Agent type=VLM2026.05 | 79 | 88 | 93 | 90 | 90 | 74 | 86 | |
| Idefics2 8B*Supervision=No access to text env, Agent type=VLM2026.05 | 75 | 77 | 74 | 69 | 67 | 50 | 69 | |
| ReActSupervision=Access to text env, Agent type=LLM2026.05 | 71 | 28 | 65 | 62 | 44 | 35 | 54 | |
| EMMASupervision=Access to text env, Agent type=VLM2026.05 | 71 | 88 | 94 | 85 | 83 | 67 | 82 | |
| RL4VLMSupervision=No access to text env, Agent type=VLM2026.05 | 47 | 14 | 10 | 14 | 18 | 18 | 21 | |
| MiniGPT-4Supervision=No access to text env, Agent type=VLM2026.05 | 4 | 17 | 0 | 19 | 17 | 6 | 16 |