Partially Observable Decision Making on ALFWorld
96Pick Success RateGiGPOw/o std
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GiGPOw/o stdParadigm=RL, Backbone=Qwen2.5-1.5B-Instruct, Number of seeds=32026.05 | 96 | 76.5 | 91.8 | 91.3 | 71.7 | 79.5 | 86.1 | |
| GiGPOw/stdParadigm=RL, Backbone=Qwen2.5-1.5B-Instruct, Number of seeds=32026.05 | 94.4 | 67.5 | 94.8 | 94.4 | 79.8 | 76.4 | 86.7 | |
| Gemini-2.5-ProParadigm=Prompt, Evaluation Protocol=zero-shot reference2026.05 | 92.8 | 63.3 | 62.1 | 69 | 26.6 | 58.7 | 60.3 | |
| ReBelParadigm=RL, Backbone=Qwen2.5-1.5B-Instruct, Number of seeds=3, Initialization=SFT warm start2026.05 | 91.8 | 84.2 | 91.3 | 95.8 | 84.8 | 96.5 | 93.2 | |
| RLOOParadigm=RL, Backbone=Qwen2.5-1.5B-Instruct, Number of seeds=32026.05 | 88.3 | 52.8 | 71 | 62.8 | 66.4 | 56.9 | 69.7 | |
| GRPOParadigm=RL, Backbone=Qwen2.5-1.5B-Instruct, Number of seeds=32026.05 | 85.3 | 53.7 | 84.5 | 78.2 | 59.7 | 53.5 | 72.8 | |
| GPT-4oParadigm=Prompt, Evaluation Protocol=zero-shot reference2026.05 | 75.3 | 60.8 | 31.2 | 56.7 | 21.6 | 49.8 | 48 | |
| PPOParadigm=RL, Backbone=Qwen2.5-1.5B-Instruct, Number of seeds=32026.05 | 64.8 | 40.5 | 57.1 | 60.6 | 46.4 | 47.4 | 54.4 | |
| ReflexionParadigm=Prompt, Backbone=Qwen2.5-1.5B-Instruct2026.05 | 35.3 | 22.2 | 21.7 | 13.6 | 19.4 | 3.7 | 21.8 | |
| ReActParadigm=Prompt, Backbone=Qwen2.5-1.5B-Instruct2026.05 | 17.4 | 20.5 | 15.7 | 6.2 | 7.7 | 2 | 12.8 | |
| Qwen2.5Paradigm=Prompt, Backbone=Qwen2.5-1.5B-Instruct2026.05 | 5.9 | 5.5 | 3.3 | 9.7 | 4.2 | 0 | 4.1 |