Sub-Goal Completion on AppWorld 1.0 (test_normal)
97.2Sub-Goal Completion RateASSAY
Evaluation results
| Method | Links | |
|---|---|---|
| ASSAYModel=Sonnet 4.52026.06 | 97.2 | |
| ASSAYModel=GPT-5.42026.06 | 96.7 | |
| ASSAYModel=Gemini 2.52026.06 | 95.5 | |
| ReActModel=Sonnet 4.52026.06 | 94 | |
| ReActModel=GPT-5.42026.06 | 91.9 | |
| ReActModel=Gemini 2.52026.06 | 87.8 | |
| (Qwen3-14B, wt-tuned)Model=Leaderboard best*2026.06 | 80.4 | |
| ASSAYModel=GPT-5.12026.06 | 71.4 | |
| ASSAYModel=DeepSeek-V32026.06 | 71.4 | |
| ACEModel=DeepSeek-V32026.06 | 66.1 | |
| CUGAModel=GPT-4.12026.06 | 62.5 | |
| ASSAYModel=GPT-4.12026.06 | 60.7 | |
| ASSAYModel=GPT-4o2026.06 | 58.9 | |
| Gupta et al.Model=GPT-4o2026.06 | 57.1 | |
| ACEModel=GPT-5.12026.06 | 55.4 | |
| ReActModel=DeepSeek-V32026.06 | 48.2 | |
| ReActModel=GPT-5.12026.06 | 46.4 | |
| ReActModel=GPT-4.12026.06 | 46.4 | |
| ReActModel=GPT-4o2026.06 | 32.1 |