Agentic Task Completion on AppWorld LeaderBoard
48.8Greedy Success RateReAct
Evaluation Results
| Method | Links | |
|---|---|---|
| ReActThink=false, LLM=GPT-4o, Params=/2025.12 | 48.8 | |
| CuESThink=false, LLM=Qwen2.5, Params=14B2025.12 | 45.24 | |
| PlanExecThink=false, LLM=GPT-4o, Params=/2025.12 | 44.6 | |
| FullCodeReflThink=false, LLM=GPT-4o, Params=/2025.12 | 33.9 | |
| PlanExecThink=false, LLM=GPT-4 Turbo, Params=/2025.12 | 32.7 | |
| ReActThink=false, LLM=GPT-4 Turbo, Params=/2025.12 | 26.8 | |
| FullCodeReflThink=false, LLM=GPT-4 Turbo, Params=/2025.12 | 25.6 | |
| FullCodeReflThink=false, LLM=LLaMA3, Params=70B2025.12 | 24.4 | |
| ReActThink=false, LLM=DeepSeekCoder, Params=33B2025.12 | 20.8 | |
| FullCodeReflThink=false, LLM=DeepSeekCoder, Params=33B2025.12 | 13.1 | |
| PlanExecThink=false, LLM=LLaMA3, Params=70B2025.12 | 8.9 | |
| ReActThink=false, LLM=LLaMA3, Params=70B2025.12 | 7.1 | |
| PlanExecThink=false, LLM=DeepSeekCoder, Params=33B2025.12 | 1.8 |