Interactive environment task execution on AppWorld normal (test)
4,554Avg@8 SuccessCuES
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CuESParams=14B2025.12 | 4,554 | 4,524 | — | — | |
| Qwen2.5Params=32B2025.12 | 3,316 | 3,473 | — | — | |
| Qwen3Params=14B2025.12 | 3,098 | 2,848 | — | — | |
| Qwen3Params=32B2025.12 | 2,879 | 3,212 | — | — | |
| Qwen3Params=8B2025.12 | 1,776 | 2,143 | — | — | |
| Qwen2.5Params=14B2025.12 | 1,176 | 1,429 | — | — | |
| Qwen3Params=4B2025.12 | 300 | 325 | — | — | |
| Qwen2.5Params=7B2025.12 | 125 | 187 | — | — | |
| ReAct + ACEBase LLM=GPT-5.1, Adaptation mode=Online Adaptation, GT Labels=✗2025.10 | 65.8 | — | 72.6 | 58.9 | |
| ReAct + ACEBase LLM=GPT-5.1, Adaptation mode=Offline Adaptation, GT Labels=✗2025.10 | 61.3 | — | 67.3 | 55.4 | |
| ReAct + ACEBase LLM=GPT-5.1, Adaptation mode=Offline Adaptation, GT Labels=✓2025.10 | 60.2 | — | 66.7 | 53.6 | |
| ReAct + DC (CU)Base LLM=GPT-5.1, Adaptation mode=Online Adaptation, GT Labels=✗2025.10 | 58.9 | — | 62.5 | 55.4 | |
| ReAct + GEPABase LLM=GPT-5.1, Adaptation mode=Offline Adaptation, GT Labels=✓2025.10 | 56.2 | — | 64.3 | 48.2 | |
| ReActBase LLM=GPT-5.1, Adaptation mode=None2025.10 | 54.2 | — | 61.9 | 46.4 | |
| Qwen2.5Params=3B2025.12 | 0 | 23 | — | — | |
| BaselineMemory Configuration=Baseline2026.03 | — | — | 69.6 | 50 | |
| Trajectory-Informed Memory GenerationMemory Configuration=+Memory2026.03 | — | — | 73.2 | 64.3 |