Household Simulation on ALFWorld (out-of-distribution)
75Put Success RateDuSAR
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DuSARModel=Llama3.1-70B2025.12 | 75 | 61.1 | 19.4 | 13 | 14.3 | 52.9 | 37.3 | |
| DuSARModel=Gemma3-12B2025.12 | 45.8 | 27.8 | 12.9 | 13 | 4.8 | 11.8 | 19.4 | |
| DuSARModel=Llama3.1-8B2025.12 | 37.5 | 22.2 | 0 | 4.3 | 0 | 5.9 | 11.7 | |
| SynapseModel=Gemma3-12B2025.12 | 12.5 | 5.5 | 16.1 | 47.8 | 4.8 | 0 | 14.5 | |
| SynapseModel=Llama3.1-70B2025.12 | 12.5 | 16.7 | 12.9 | 26.1 | 9.5 | 0 | 13 | |
| TRADModel=Llama3.1-70B2025.12 | 12.5 | 11.1 | 3.2 | 8.7 | 23.8 | 0 | 9.9 | |
| TRADModel=Gemma3-12B2025.12 | 8.3 | 11.1 | 0 | 34.8 | 0 | 0 | 9 | |
| DuSARModel=Llama2-7B2025.12 | 4.2 | 0 | 0 | 0 | 0 | 0 | 0.75 | |
| SynapseModel=Llama3.1-8B2025.12 | 4.2 | 16.7 | 0 | 0 | 0 | 0 | 3.5 | |
| SynapseModel=Llama2-7B2025.12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| TRADModel=Llama2-7B2025.12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| TRADModel=Llama3.1-8B2025.12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |