Sub-Goal Completion on AppWorld 1.0 (test_challenge)
95.4SGCASSAY
Evaluation results
| Method | Links | |
|---|---|---|
| ASSAYModel=GPT-5.42026.06 | 95.4 | |
| ASSAYModel=Sonnet 4.52026.06 | 93.5 | |
| ASSAYModel=Gemini 2.52026.06 | 91.8 | |
| ReActModel=GPT-5.42026.06 | 91.7 | |
| ReActModel=Sonnet 4.52026.06 | 88.7 | |
| ReActModel=Gemini 2.52026.06 | 80.5 | |
| ASSAYModel=DeepSeek-V32026.06 | 52.5 | |
| (Qwen3-14B, wt-tuned)Model=Leaderboard best*2026.06 | 50.4 | |
| ASSAYModel=GPT-5.12026.06 | 48.9 | |
| CUGAModel=GPT-4.12026.06 | 48.2 | |
| ASSAYModel=GPT-4.12026.06 | 48.2 | |
| ACEModel=DeepSeek-V32026.06 | 43.9 | |
| ReActModel=GPT-4.12026.06 | 32.4 | |
| ReActModel=GPT-5.12026.06 | 30.9 | |
| ASSAYModel=GPT-4o2026.06 | 30.2 | |
| ACEModel=GPT-5.12026.06 | 28.8 | |
| ReActModel=DeepSeek-V32026.06 | 25.2 | |
| Gupta et al.Model=GPT-4o2026.06 | 23 | |
| ReActModel=GPT-4o2026.06 | 13 |