Tool Use on StableToolBench
72.8I2 Category SuccessReAct+PLAY2PROMPT
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ReAct+PLAY2PROMPTBase Model=LLaMA-70B2025.03 | 72.8 | 67.5 | 73.6 | 64.1 | 66.5 | 70.7 | 69.2 | — | — | — | — | — | — | — | — | — | |
| ReAct+EasyToolBase Model=LLaMA-70B2025.03 | 70.6 | 59 | 70.3 | 63.3 | 67.6 | 63 | 65.6 | — | — | — | — | — | — | — | — | — | |
| ReAct+DRAFTBase Model=LLaMA-70B2025.03 | 68.5 | — | — | — | — | 62.3 | — | — | — | — | — | — | — | — | — | — | |
| ReAct+EasyToolBase Model=GPT-3.52025.03 | 68.3 | 57.3 | 61.4 | 69.5 | 61.5 | 60.7 | 63.1 | — | — | — | — | — | — | — | — | — | |
| ReAct+DRAFTBase Model=GPT-3.52025.03 | 68.3 | — | — | — | — | 56.1 | — | — | — | — | — | — | — | — | — | — | |
| ReAct+PLAY2PROMPTBase Model=GPT-3.52025.03 | 68.3 | 61.1 | 66.6 | 66.2 | 67 | 66.3 | 65.9 | — | — | — | — | — | — | — | — | — | |
| MoRAgent-LlamaSetting=DFS, Base Model=Llama3.2-1B-Instruct, Total Params=1.24B+0.16B2025.12 | 68.2 | 54.6 | 53.2 | 45.5 | 46.8 | 58.8 | — | 44.8 | 32.9 | 40.5 | 36.8 | 46.8 | 39.4 | 54.5 | 40.2 | — | |
| ReActBase Model=GPT-4o2025.03 | 65.3 | 54 | 69.7 | 66.1 | 59.8 | 61.7 | 62.8 | — | — | — | — | — | — | — | — | — | |
| ReAct+EasyToolBase Model=GPT-4o2025.03 | 65.3 | 49.9 | 69 | 65.2 | 61.3 | 52.5 | 60.5 | — | — | — | — | — | — | — | — | — | |
| ReAct+EasyToolBase Model=LLaMA-8B2025.03 | 64.2 | 54.7 | 58.9 | 57.9 | 56.1 | 48.1 | 56.7 | — | — | — | — | — | — | — | — | — | |
| ReAct+PLAY2PROMPTBase Model=GPT-4o2025.03 | 63.7 | 60.7 | 68.7 | 71.9 | 58.8 | 65.6 | 64.9 | — | — | — | — | — | — | — | — | — | |
| ReAct+PLAY2PROMPTBase Model=LLaMA-8B2025.03 | 63.5 | 56.6 | 65.7 | 60.9 | 62.5 | 63.8 | 62.2 | — | — | — | — | — | — | — | — | — | |
| ReActBase Model=GPT-3.52025.03 | 63.4 | 56 | 64.6 | 67.4 | 56 | 57.4 | 60.7 | — | — | — | — | — | — | — | — | — | |
| ReActBase Model=LLaMA-70B2025.03 | 63.2 | 58.4 | 68.3 | 61.1 | 63.1 | 64.3 | 63.1 | — | — | — | — | — | — | — | — | — | |
| ReAct+DRAFTBase Model=LLaMA-8B2025.03 | 62.6 | — | — | — | — | 57.9 | — | — | — | — | — | — | — | — | — | — | |
| ReAct+DRAFTBase Model=GPT-4o2025.03 | 62.1 | — | — | — | — | 63.9 | — | — | — | — | — | — | — | — | — | — | |
| ReActBase Model=LLaMA-8B2025.03 | 61.3 | 50.6 | 59.3 | 53.2 | 58 | 52.3 | 55.8 | — | — | — | — | — | — | — | — | — | |
| ToolLLaMA-v2-7BSetting=DFS2025.12 | 60.3 | 61 | 58.8 | 45.6 | 53.5 | 48.1 | — | 43.6 | 35.8 | 41.3 | 34.2 | 43.8 | 45.1 | 54.6 | 40.6 | — | |
| MoRAgent-PhiSetting=DFS, Base Model=Phi3.5-mini-Instruct, Total Params=3.82B+0.36B2025.12 | 59.7 | 55.9 | 60.9 | 56.6 | 55.4 | 63.6 | — | 48.5 | 41.1 | 47.1 | 44.3 | 48.4 | 50.8 | 58.7 | 46.7 | — | |
| GPT4Setting=DFS2025.12 | 55.6 | 59.2 | 61.7 | 65.7 | 55.2 | 66.1 | — | 50.1 | 44.2 | 57.9 | 41.2 | 53.7 | 45.4 | 60.6 | 48.8 | — | |
| MoRAgent-PhiSetting=CoT, Base Model=Phi3.5-mini-Instruct, Total Params=3.82B+0.36B2025.12 | 55 | 51.8 | 55.4 | 54.4 | 48.1 | 62.3 | — | 47.2 | 38.6 | 51 | 33 | 45.2 | 42.6 | 54.5 | 42.9 | — | |
| GPT4Setting=CoT2025.12 | 51.9 | 52.8 | 56.6 | 51.9 | 52.8 | 52.5 | — | 48.4 | 40.3 | 54.2 | 36.6 | 47.6 | 45.2 | 53.1 | 45.4 | — | |
| ToolLLaMA-v2-7BSetting=CoT2025.12 | 51.6 | 51.8 | 53.1 | 46.4 | 48.9 | 37.2 | — | 39.2 | 33.1 | 39.9 | 31.3 | 40.7 | 42.2 | 48.2 | 37.7 | — | |
| MoRAgent-LlamaSetting=CoT, Base Model=Llama3.2-1B-Instruct, Total Params=1.24B+0.16B2025.12 | 42.6 | 54.9 | 53.1 | 58.2 | 38.1 | 47.9 | — | 40.5 | 38 | 39.9 | 34 | 41.1 | 29.5 | 49.1 | 37.2 | — | |
| BaseSetting=DFS, Base Model=Phi3.5-mini-Instruct, Base Model Params=3.82B2025.12 | 41.5 | 46.4 | 51.1 | 50.6 | 39.9 | 37.6 | — | 35 | 30.4 | 49 | 30.2 | 27.4 | 32.8 | 44.5 | 34.1 | — | |
| BaseSetting=CoT, Base Model=Phi3.5-mini-Instruct, Base Model Params=3.82B2025.12 | 26.7 | 29.7 | 36 | 37.9 | 20.9 | 17.8 | — | 30.7 | 30.4 | 34.6 | 17 | 25 | 15.8 | 28.2 | 25.6 | — | |
| BaseSetting=CoT, Base Model=Llama3.2-1B-Instruct, Base Model Params=1.24B2025.12 | 5.9 | 14.4 | 12.2 | 10.1 | 3.9 | 8.2 | — | 23.3 | 15.8 | 15 | 11.3 | 11.3 | 13.1 | 9.1 | 15 | — | |
| BaseSetting=DFS, Base Model=Llama3.2-1B-Instruct, Base Model Params=1.24B2025.12 | 2.3 | 11.7 | 14.6 | 11.3 | 8.5 | 11.5 | — | 20.8 | 17.1 | 18.3 | 9.4 | 12.9 | 11.5 | 10 | 15 | — | |
| COMAPBackbone=Qwen3-4B, Category=Co-evolving, Training-based=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 56 | |
| COMAPBackbone=Qwen3-8B, Category=Co-evolving, Training-based=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 70.5 | |
| Imagine-and-ActBackbone=GPT-5.4, Category=API-Called Models, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.2 | |
| Imagine-and-ActBackbone=DeepSeek-V4-Flash, Category=API-Called Models, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 58.3 | |
| Imagine-and-ActBackbone=DeepSeek-V4-Pro, Category=API-Called Models, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 65.2 | |
| Imagine-and-ActBackbone=Qwen3-4B, Category=Base Agent Planning, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24 | |
| Imagine-and-ActBackbone=Qwen3-8B, Category=Base Agent Planning, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.5 | |
| Imagine-and-Act (w/ SFT)Backbone=Qwen3-4B, Category=Base Agent Planning, Training-based=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39 | |
| Imagine-and-Act (w/ SFT)Backbone=Qwen3-8B, Category=Base Agent Planning, Training-based=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 46.5 | |
| ITPIBackbone=Qwen3-4B, Category=Test-time Evolving, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 30 | |
| ITPIBackbone=Qwen3-8B, Category=Test-time Evolving, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35 | |
| ITPRBackbone=Qwen3-4B, Category=World Modeling, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 48 | |
| ITPRBackbone=Qwen3-8B, Category=World Modeling, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 68 | |
| IWMBackbone=Qwen3-4B, Category=World Modeling, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.5 | |
| IWMBackbone=Qwen3-8B, Category=World Modeling, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 44 | |
| RAPBackbone=Qwen3-4B, Category=Test-time Evolving, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25 | |
| RAPBackbone=Qwen3-8B, Category=Test-time Evolving, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28 | |
| ReActBackbone=GPT-5.4, Category=API-Called Models, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 66.8 | |
| ReActBackbone=DeepSeek-V4-Flash, Category=API-Called Models, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 54.8 | |
| ReActBackbone=DeepSeek-V4-Pro, Category=API-Called Models, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 62.5 | |
| ReActBackbone=Qwen3-4B, Category=Base Agent Planning, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 21.5 | |
| ReActBackbone=Qwen3-8B, Category=Base Agent Planning, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 26 | |
| ReAct (w/ SFT)Backbone=Qwen3-4B, Category=Base Agent Planning, Training-based=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35.5 | |
| ReAct (w/ SFT)Backbone=Qwen3-8B, Category=Base Agent Planning, Training-based=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 42 | |
| ReflexionBackbone=Qwen3-4B, Category=Test-time Evolving, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.5 | |
| ReflexionBackbone=Qwen3-8B, Category=Test-time Evolving, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.5 | |
| WKMBackbone=Qwen3-4B, Category=World Modeling, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 35 | |
| WKMBackbone=Qwen3-8B, Category=World Modeling, Training-free=true2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.5 |