Agent Task on AppWorld Normal (test)
92.3TGCACCORD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACCORDBackbone=Claude-4.5-sonnet2026.06 | 92.3 | 87.5 | |
| ReActBackbone=Claude-4.5-sonnet2026.06 | 88.1 | 83.9 | |
| ACCORDBackbone=Qwen3.5-27B-FP82026.06 | 81.5 | 67.9 | |
| ACCORDBackbone=GPT-5-mini2026.06 | 78 | 62.5 | |
| ReAct + ACEBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✓2025.10 | 76.2 | 64.3 | |
| ReAct + ACEBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✗2025.10 | 75 | 64.3 | |
| Claude-Sonnet-4.5CoEvolve=false2026.04 | 73.81 | 55.36 | |
| ReActBackbone=Qwen3.5-27B-FP82026.06 | 71.4 | 51.8 | |
| ReAct + ACEBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Online Adaptation, GT Labels=✗2025.10 | 69.6 | 53.6 | |
| ReAct + DC (CU)Base LLM=DeepSeek-V3.1-671B, Adaptation Mode=Online Adaptation, GT Labels=✗2025.10 | 65.5 | 58.9 | |
| ReAct + GEPABase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✓2025.10 | 64.9 | 44.6 | |
| ReAct + ICLBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=Offline Adaptation, GT Labels=✓2025.10 | 64.3 | 46.4 | |
| ReActBase LLM=DeepSeek-V3.1-671B, Adaptation Mode=None2025.10 | 63.7 | 42.9 | |
| ReActBackbone=GPT-5-mini2026.06 | 63.7 | 46.4 | |
| Self-RefineBackbone=GPT-5-mini2026.06 | 60.1 | 39.3 | |
| Qwen3-30B-A3BCoEvolve=true2026.04 | 54.76 | 33.93 | |
| Gemini-2.5-FlashCoEvolve=false2026.04 | 53.57 | 32.14 | |
| DeepSeek-V3.2CoEvolve=false2026.04 | 47.02 | 30.36 | |
| FullCodeReflexBackbone=GPT-5-mini2026.06 | 43.5 | 30.4 | |
| Qwen3-4B-InstructCoEvolve=true2026.04 | 35.71 | 14.28 | |
| Qwen3-30B-A3BCoEvolve=false2026.04 | 31.55 | 12.5 | |
| LLaMA-3.3-70BCoEvolve=false2026.04 | 31.54 | 8.92 | |
| GPT-4CoEvolve=false2026.04 | 30.4 | 21.4 | |
| Qwen2.5-7B-InstructCoEvolve=true2026.04 | 27.98 | 12.5 | |
| Gemma-3-27BCoEvolve=false2026.04 | 23.81 | 7.14 | |
| ACE (online)Method Category=Cross-trajectory memory methods2026.06 | 21.8 | 10.2 | |
| GPT-OSS-20BCoEvolve=false2026.04 | 17.86 | 3.57 | |
| Qwen3-4B-InstructCoEvolve=false2026.04 | 16.67 | 5.36 | |
| Qwen2.5-7B-InstructCoEvolve=false2026.04 | 1.19 | 0 |