Agentic Task Completion on AppWorld Challenge (test)
49.88Task Goal Completion (TGC)Claude-Sonnet-4.5
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Claude-Sonnet-4.5CoEvolve=false2026.04 | 49.88 | 32.37 | |
| Gemini-2.5-FlashCoEvolve=false2026.04 | 40.05 | 20.14 | |
| Qwen3-30B-A3BCoEvolve=true2026.04 | 31.65 | 16.55 | |
| DeepSeek-V3.2CoEvolve=false2026.04 | 22.78 | 9.35 | |
| Qwen3-30B-A3BCoEvolve=false2026.04 | 19.9 | 5.76 | |
| Qwen3-4B-InstructCoEvolve=true2026.04 | 17.03 | 6.47 | |
| LLaMA-3.3-70BCoEvolve=false2026.04 | 16.31 | 4.32 | |
| GPT-4CoEvolve=false2026.04 | 14.6 | 9.3 | |
| Gemma-3-27BCoEvolve=false2026.04 | 9.83 | 0.72 | |
| Qwen2.5-7B-InstructCoEvolve=true2026.04 | 8.39 | 2.16 | |
| Qwen3-4B-InstructCoEvolve=false2026.04 | 7.91 | 2.16 | |
| GPT-OSS-20BCoEvolve=false2026.04 | 5.52 | 0 | |
| Qwen2.5-7B-InstructCoEvolve=false2026.04 | 0.72 | 0 |