Tool Use on API-Bank (test)
75.54Overall AccuracyCAHL
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CAHL2026.06 | 75.54 | — | 81.7 | 64.2 | 62.6 | — | — | |
| TUMIX2026.06 | 73.7 | — | 76.19 | 53.73 | 76.34 | — | — | |
| ToolSample2026.06 | 64.99 | — | 73.2 | 65.7 | 39.7 | — | — | |
| GD2PO-HardModel=Qwen2.5-3B-Instruct2026.06 | 62.88 | — | 69.27 | 57.61 | 46.11 | 1 | 1.629 | |
| GD2PO-SNRModel=Qwen2.5-3B-Instruct2026.06 | 62.35 | — | 68.57 | 58.21 | 45.49 | 1 | 1.624 | |
| ToolRL2026.06 | 61.81 | — | 72.2 | 56.7 | 32.8 | — | — | |
| GDPOModel=Qwen2.5-3B-Instruct2026.06 | 61.57 | — | 67.42 | 56.12 | 46.57 | 1 | 1.616 | |
| GRPOModel=Qwen2.5-3B-Instruct2026.06 | 61.24 | — | 67.22 | 56.42 | 45.5 | 0.99 | 1.602 | |
| Tool-N12026.06 | 60.47 | — | 70.7 | 46.3 | 36.6 | — | — | |
| GD2PO-HardModel=Llama3.1-8B-Instruct2026.06 | 58.79 | — | 64.81 | 43.88 | 48.09 | 1 | 1.588 | |
| GD2PO-SNRModel=Llama3.1-8B-Instruct2026.06 | 58.79 | — | 64.11 | 42.39 | 51 | 1 | 1.588 | |
| QwenBase Model=Qwen-2.5-Instruct2026.06 | 58.29 | — | 65.4 | 43.3 | 44.3 | — | — | |
| GRPOModel=Llama3.1-8B-Instruct2026.06 | 57.05 | — | 63.91 | 44.78 | 42.44 | 1 | 1.571 | |
| GDPOModel=Llama3.1-8B-Instruct2026.06 | 55.44 | — | 60.95 | 42.69 | 45.19 | 1 | 1.554 | |
| EASYTool2026.06 | 54.61 | — | 61.15 | 35.82 | 44.27 | — | — | |
| GD2PO-HardModel=Qwen2.5-1.5B-Instruct2026.06 | 52.96 | — | 59.95 | 44.48 | 36.03 | 0.99 | 1.52 | |
| GD2PO-SNRModel=Qwen2.5-1.5B-Instruct2026.06 | 52.36 | — | 58.59 | 45.97 | 36.64 | 1 | 1.524 | |
| GRPOModel=Qwen2.5-1.5B-Instruct2026.06 | 51.46 | — | 56.59 | 43.28 | 40 | 0.99 | 1.505 | |
| GDPOModel=Qwen2.5-1.5B-Instruct2026.06 | 50.89 | — | 56.69 | 44.78 | 36.33 | 1 | 1.509 | |
| AvaTaR2026.03 | — | 63.5 | — | — | — | — | — | |
| Bloomberg AI Engineering2026.03 | — | 66.7 | — | — | — | — | — | |
| Claude 3.5 Sonnetevaluation_mode=zero-shot2024.07 | — | 92.6 | — | — | — | — | — | |
| Gemma 2 9Bevaluation_mode=zero-shot2024.07 | — | 56.5 | — | — | — | — | — | |
| Gorilla2026.03 | — | 38.7 | — | — | — | — | — | |
| GPT-3.5 Turboevaluation_mode=zero-shot2024.07 | — | 60.9 | — | — | — | — | — | |
| GPT-4evaluation_mode=zero-shot2024.07 | — | 89 | — | — | — | — | — | |
| GPT-4 (function calling)2026.03 | — | 67.1 | — | — | — | — | — | |
| GPT-4oevaluation_mode=zero-shot2024.07 | — | 91.3 | — | — | — | — | — | |
| Llama 3 405Bevaluation_mode=zero-shot2024.07 | — | 92.3 | — | — | — | — | — | |
| Llama 3 70Bevaluation_mode=zero-shot2024.07 | — | 90 | — | — | — | — | — | |
| Llama 3 8Bevaluation_mode=zero-shot2024.07 | — | 82.6 | — | — | — | — | — | |
| Mistral 7Bevaluation_mode=zero-shot2024.07 | — | 55.8 | — | — | — | — | — | |
| Mixtral 8x22Bevaluation_mode=zero-shot2024.07 | — | 73.1 | — | — | — | — | — | |
| ToolLLM2026.03 | — | 52.4 | — | — | — | — | — | |
| ToolRLABackbone=Qwen3-14B2026.03 | — | 71.8 | — | — | — | — | — |