Agent Task on AppWorld Challenge (test)
84.2Task Goal Completion (TGC)ACCORD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACCORDBackbone=Claude-4.5-sonnet2026.06 | 84.2 | 70.5 | |
| ReActBackbone=Claude-4.5-sonnet2026.06 | 73.4 | 59.7 | |
| ACCORDBackbone=Qwen3.5-27B-FP82026.06 | 69.8 | 53.2 | |
| ReAct + ACEBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Online Adaptation, GT Labels=✗2025.10 | 66 | 48.9 | |
| ReActBackbone=Qwen3.5-27B-FP82026.06 | 63.1 | 46.8 | |
| ACCORDBackbone=GPT-5-mini2026.06 | 62.6 | 38.9 | |
| ReAct + ACEBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✓2025.10 | 57.3 | 39.6 | |
| ReAct + ACEBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✗2025.10 | 54.4 | 35.2 | |
| ReAct + DC (CU)Base LLM=DeepSeek-V3.1-671B, Adaptation Mode=Online Adaptation, GT Labels=✗2025.10 | 52.3 | 30.8 | |
| ReAct + ICLBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✓2025.10 | 46 | 27.3 | |
| ReAct + GEPABase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✓2025.10 | 46 | 30.2 | |
| ReAct + ACEBase LLM=GPT-OSS-120B, Adaptation Mode=Online Adaptation, GT Labels=false2025.10 | 43.2 | 20.1 | |
| ReActBackbone=GPT-5-mini2026.06 | 42 | 20.9 | |
| ReActBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=None2025.10 | 41.5 | 21.6 | |
| ReAct + ACEBase LLM=GPT-OSS-120B, Adaptation Mode=Offline Adaptation, GT Labels=true2025.10 | 40.3 | 20.9 | |
| ReAct + GEPABase LLM=GPT-OSS-120B, Adaptation Mode=Offline Adaptation, GT Labels=true2025.10 | 40.1 | 20.9 | |
| Self-RefineBackbone=GPT-5-mini2026.06 | 40.1 | 17.3 | |
| ReAct + ACEBase LLM=GPT-OSS-120B, Adaptation Mode=Offline Adaptation, GT Labels=false2025.10 | 39.6 | 18.7 | |
| ReActBase LLM=GPT-OSS-120B, Adaptation Mode=Base LLM2025.10 | 34.5 | 15.1 | |
| ReAct + DC (CU)Base LLM=GPT-OSS-120B, Adaptation Mode=Online Adaptation, GT Labels=false2025.10 | 30.8 | 18.2 | |
| ACE (online)Method Category=Cross-trajectory memory methods2026.06 | 7.1 | 6.4 |