Tool-use on ToolBench (I1/I2/I3 Evaluation)
80.67Average Pass RateQwen2.5-7B-Instruct-CAST
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct-CASTCategory=Open Source2026.05 | 80.67 | — | — | — | — | — | — | — | — | — | — | — | — | 79.43 | |
| Qwen2.5-Coder-7B-Instruct-CASTCategory=Open Source2026.05 | 80.61 | — | — | — | — | — | — | — | — | — | — | — | — | 79.14 | |
| Llama-3.2-8B-Instruct-CASTCategory=Open Source2026.05 | 75.93 | — | — | — | — | — | — | — | — | — | — | — | — | 47.23 | |
| Qwen2.5-7B-Instruct-GRPOCategory=Open Source2026.05 | 72.71 | — | — | — | — | — | — | — | — | — | — | — | — | 68.23 | |
| Attention Buckets + DFSDT-RetrieverModel=ToolLlama, Parameters=7B2023.12 | 71.3 | 68.5 | 65 | 70 | 65.5 | 65 | 67 | 84 | 78 | 71 | 64.5 | 69 | 89 | 71.5 | |
| Llama-3.2-8B-Instruct-GRPOCategory=Open Source2026.05 | 71.23 | — | — | — | — | — | — | — | — | — | — | — | — | 45.23 | |
| Qwen2.5-Coder-7B-Instruct-GRPOCategory=Open Source2026.05 | 71.13 | — | — | — | — | — | — | — | — | — | — | — | — | 67.82 | |
| DFSDTModel=GPT42023.12 | 71.1 | 60 | 67.5 | 71.5 | 67.8 | 67 | 66.5 | 79.5 | 73.3 | 77.5 | 63.3 | 71 | 84 | 70.4 | |
| Qwen-MaxCategory=Closed Source2026.05 | 70.93 | — | — | — | — | — | — | — | — | — | — | — | — | 71.83 | |
| DeepSeek-V3Category=Closed Source2026.05 | 70.73 | — | — | — | — | — | — | — | — | — | — | — | — | 72.93 | |
| Llama-3.2-8B-Instruct-SFTCategory=Open Source2026.05 | 70.63 | — | — | — | — | — | — | — | — | — | — | — | — | 45.83 | |
| Qwen2.5-7B-Instruct-SFTCategory=Open Source2026.05 | 68.67 | — | — | — | — | — | — | — | — | — | — | — | — | 65.23 | |
| GraniteCategory=Other2026.05 | 68.47 | — | — | — | — | — | — | — | — | — | — | — | — | 50.17 | |
| Attention Buckets + DFSDTModel=ToolLlama, Parameters=7B2023.12 | 68 | 66.5 | 67.5 | 61.5 | 62 | 62 | 65.5 | 78 | 71.5 | 73 | 66.5 | 67 | 82 | 69.2 | |
| Qwen2.5-Coder-7B-Instruct-SFTCategory=Open Source2026.05 | 67.84 | — | — | — | — | — | — | — | — | — | — | — | — | 65.37 | |
| DFSDT-RetrieverModel=ToolLlama2023.12 | 67.3 | 64 | 62.3 | 64 | 59 | 60.5 | 55.5 | 81.5 | 68.5 | 68.5 | 60.8 | 65 | 73 | 63.1 | |
| DFSDTModel=ToolLlama2023.12 | 66.7 | 57 | 55 | 61 | 55.3 | 62 | 54.5 | 77 | 68.5 | 77 | 58 | 66 | 69 | 60 | |
| OTCCategory=Other2026.05 | 65.49 | — | — | — | — | — | — | — | — | — | — | — | — | 36.12 | |
| DFSDTModel=ChatGPT2023.12 | 64.8 | 54.5 | 60.5 | 65 | 62 | 60.5 | 57.3 | 75 | 72 | 71.5 | 64.8 | 62 | 69 | 64.3 | |
| GPT-4oCategory=Closed Source2026.05 | 64.43 | — | — | — | — | — | — | — | — | — | — | — | — | 67.63 | |
| GorillaCategory=Other2026.05 | 62.27 | — | — | — | — | — | — | — | — | — | — | — | — | 46.29 | |
| SynWorldModel=GPT-4-turbo2025.04 | 59.33 | — | — | — | — | — | — | — | — | — | — | — | — | 73 | |
| SynWorldModel=Qwen2-72B-Instruct2025.04 | 58.52 | — | — | — | — | — | — | — | — | — | — | — | — | 73 | |
| ReACTModel=GPT42023.12 | 57.2 | 53.5 | 60 | 50 | 58.8 | 53.5 | 63.5 | 67 | 65.8 | 72 | 60.3 | 47 | 78 | 64.4 | |
| SynWorldModel=Qwen-long2025.04 | 57.2 | — | — | — | — | — | — | — | — | — | — | — | — | 81 | |
| Self-RefineModel=GPT-4-turbo2025.04 | 56.8 | — | — | — | — | — | — | — | — | — | — | — | — | 73 | |
| DRAFTModel=Qwen2-72B-Instruct2025.04 | 56.43 | — | — | — | — | — | — | — | — | — | — | — | — | 69 | |
| DRAFTModel=GPT-4-turbo2025.04 | 54.83 | — | — | — | — | — | — | — | — | — | — | — | — | 72 | |
| Self-RefineModel=Qwen2-72B-Instruct2025.04 | 54.33 | — | — | — | — | — | — | — | — | — | — | — | — | 65 | |
| DRAFTModel=Qwen-long2025.04 | 54.2 | — | — | — | — | — | — | — | — | — | — | — | — | 79 | |
| Self-RefineModel=Qwen-long2025.04 | 53.7 | — | — | — | — | — | — | — | — | — | — | — | — | 77 | |
| EasyToolModel=Qwen2-72B-Instruct2025.04 | 52.97 | — | — | — | — | — | — | — | — | — | — | — | — | 58 | |
| PORToolBackbone=Qwen-2.5-7B-Instruct2025.10 | 52.8 | 57.6 | — | 55.2 | — | 56.3 | — | 47.6 | — | 47.8 | — | 52.2 | — | — | |
| EasyToolModel=GPT-4-turbo2025.04 | 51.67 | — | — | — | — | — | — | — | — | — | — | — | — | 68 | |
| ARPOfmBackbone=Qwen-2.5-7B-Instruct2025.10 | 50.9 | 58.4 | — | 49.6 | — | 56 | — | 47.9 | — | 50.3 | — | 43.4 | — | — | |
| EasyToolModel=Qwen-long2025.04 | 50.8 | — | — | — | — | — | — | — | — | — | — | — | — | 63 | |
| ReActModel=GPT-4-turbo2025.04 | 50.67 | — | — | — | — | — | — | — | — | — | — | — | — | 67 | |
| ReActModel=Qwen2-72B-Instruct2025.04 | 49.43 | — | — | — | — | — | — | — | — | — | — | — | — | 55 | |
| GRPOfmBackbone=Qwen-2.5-7B-Instruct2025.10 | 49.4 | 55.1 | — | 50.7 | — | 57.2 | — | 38.1 | — | 46.7 | — | 48.4 | — | — | |
| ToRLBackbone=Qwen-2.5-7B-Instruct2025.10 | 49.3 | 52.7 | — | 49.1 | — | 54.5 | — | 41.7 | — | 48.5 | — | 49 | — | — | |
| SimpleTIRBackbone=Qwen-2.5-7B-Instruct2025.10 | 49.2 | 53.8 | — | 47.2 | — | 55 | — | 46.6 | — | 50.2 | — | 42.1 | — | — | |
| ToolformerCategory=Other2026.05 | 48.92 | — | — | — | — | — | — | — | — | — | — | — | — | 22.11 | |
| AgentFlowBackbone=Qwen-2.5-7B-Instruct2025.10 | 48.5 | 52.6 | — | 45.3 | — | 54.2 | — | 38.4 | — | 50.8 | — | 49.5 | — | — | |
| ReActModel=Qwen-long2025.04 | 48.3 | — | — | — | — | — | — | — | — | — | — | — | — | 71 | |
| ToolAlignCategory=Other2026.05 | 46.78 | — | — | — | — | — | — | — | — | — | — | — | — | 22.36 | |
| ReActCategory=Other2026.05 | 43.37 | — | — | — | — | — | — | — | — | — | — | — | — | 18.22 | |
| DFSDTModel=Text-Davinci-0032023.12 | 43.1 | 43.5 | 40.3 | 44 | 43.8 | 46 | 46.8 | 37 | 40.5 | 42 | 43.3 | 46 | 63 | 46.3 | |
| ReACTModel=ChatGPT2023.12 | 40.2 | 41.5 | — | 44 | — | 44.5 | — | 42.5 | — | — | — | 22 | — | — | |
| Attention Buckets + ReACTModel=ToolLlama, Parameters=7B2023.12 | 31.3 | 31.5 | 45 | 32 | 42.5 | 33.5 | 49 | 31.5 | 65 | 32 | 42 | 28 | 58 | 50.3 | |
| ReACTModel=ToolLlama2023.12 | 29 | 25 | 45 | 29 | 42 | 33 | 47.5 | 30.5 | 50.8 | 31.5 | 41.8 | 25 | 55 | 47 | |
| DFSDTModel=Claude-22023.12 | 22.6 | 20.5 | 38 | 31 | 44.3 | 18.5 | 43.3 | 17 | 36.8 | 20.5 | 33.5 | 28 | 65 | 43.5 | |
| ReACTModel=Text-Davinci-0032023.12 | 16.5 | 12 | 28.5 | 20 | 35.3 | 20 | 31 | 8.5 | 29.8 | 14.5 | 29.8 | 24 | 45 | 33.2 | |
| ReACTModel=Claude-22023.12 | 6.8 | 5.5 | 31 | 3.5 | 27.8 | 5.5 | 33.8 | 6 | 35 | 6 | 31.5 | 14 | 47.5 | 34.4 |