Tool Calling on API-Bank L-1
94.99F1 Name MatchHiTEC-KTO
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HiTEC-KTOModel=Llama3-8B2025.05 | 94.99 | — | — | 84.99 | — | |
| HiTEC-ICLModel=GPT-4-Turbo2025.05 | 94.96 | — | — | 88.44 | — | |
| HiTEC-ICLBackbone=GPT-4-Turbo2025.05 | 94.96 | — | — | 88.44 | — | |
| VanillaModel=GPT-4-Turbo2025.05 | 94.77 | — | — | 86.66 | — | |
| VanillaBackbone=GPT-4-Turbo2025.05 | 94.77 | — | — | 86.66 | — | |
| HiTEC-ICLModel=Llama3.1-70B2025.05 | 93.21 | — | — | 79.05 | — | |
| HiTEC-ICLModel=GPT-3.5-Turbo-01252025.05 | 92.38 | — | — | 84.71 | — | |
| + CoTModel=GPT-4-Turbo2025.05 | 91.7 | — | — | 84.84 | — | |
| CoTBackbone=GPT-4-Turbo2025.05 | 91.7 | — | — | 84.84 | — | |
| HiTEC-KTOModel=Llama3.1-70B2025.05 | 90.78 | — | — | 77.07 | — | |
| VanillaModel=Llama3.1-70B2025.05 | 90.15 | — | — | 76.42 | — | |
| BaselineModel=Llama3.1-70B2025.05 | 90.15 | — | — | 76.42 | — | |
| HiTEC-KTOModel=Qwen2.5-7B2025.05 | 89.38 | — | — | 81.67 | — | |
| HiTEC-KTOModel=Qwen2.5-3B2025.05 | 89.07 | — | — | 80.16 | — | |
| HiTEC-KTOModel=Qwen2.5-1.5B2025.05 | 88.92 | — | — | 79.46 | — | |
| BaselineModel=Hammer2-7B2025.05 | 88.91 | — | — | 81.28 | — | |
| BaselineModel=Hammer2-1.5B2025.05 | 88.63 | — | — | 79.26 | — | |
| BaselineModel=Hammer2-3B2025.05 | 88.63 | — | — | 79.04 | — | |
| HiTEC-KTOModel=Qwen2.5-0.5B2025.05 | 88.29 | — | — | 78.34 | — | |
| HiTEC-KTOModel=Llama3.1-8B2025.05 | 87.47 | — | — | 80.99 | — | |
| + CoTModel=Llama3.1-70B2025.05 | 87.05 | — | — | 73.64 | — | |
| HiTEC-KTOModel=Llama3-70B2025.05 | 86.88 | — | — | 79.71 | — | |
| VanillaModel=Llama3-70B2025.05 | 86.47 | — | — | 78.87 | — | |
| BaselineModel=Llama3-70B2025.05 | 86.47 | — | — | 78.87 | — | |
| + Function CallingModel=GPT-3.5-Turbo-01252025.05 | 85.57 | — | — | 82.47 | — | |
| VanillaModel=GPT-3.5-Turbo-01252025.05 | 85.26 | — | — | 77.06 | — | |
| HiTEC-ICLModel=Llama3-70B2025.05 | 84.85 | — | — | 78.97 | — | |
| + Function CallingModel=GPT-4-Turbo2025.05 | 84.03 | — | — | 81.4 | — | |
| Function CallingBackbone=GPT-4-Turbo2025.05 | 84.03 | — | — | 81.4 | — | |
| + CoTModel=Llama3-70B2025.05 | 78.41 | — | — | 74.52 | — | |
| HiTEC-ICLModel=Llama3-8B2025.05 | 75.93 | — | — | 70.55 | — | |
| HiTEC-ICLBackbone=Llama3-8B2025.05 | 75.93 | — | — | 70.55 | — | |
| HiTEC-ICLModel=Llama3.1-8B2025.05 | 75.07 | — | — | 64.61 | — | |
| HiTEC-ICLBackbone=Llama3.1-8B2025.05 | 75.07 | — | — | 64.61 | — | |
| VanillaModel=Llama3.1-8B2025.05 | 73.06 | — | — | 63.62 | — | |
| BaselineModel=Llama3.1-8B2025.05 | 73.06 | — | — | 63.62 | — | |
| VanillaBackbone=Llama3.1-8B2025.05 | 73.06 | — | — | 63.62 | — | |
| BaselineModel=Hammer2-0.5B2025.05 | 71.2 | — | — | 59.03 | — | |
| VanillaModel=Llama3-8B2025.05 | 69.19 | — | — | 67.73 | — | |
| BaselineModel=Llama3-8B2025.05 | 69.19 | — | — | 67.73 | — | |
| VanillaBackbone=Llama3-8B2025.05 | 69.19 | — | — | 67.73 | — | |
| + CoTModel=GPT-3.5-Turbo-01252025.05 | 65.28 | — | — | 59.13 | — | |
| + CoTModel=Llama3.1-8B2025.05 | 65.22 | — | — | 61.61 | — | |
| CoTBackbone=Llama3.1-8B2025.05 | 65.22 | — | — | 61.61 | — | |
| + CoTModel=Llama3-8B2025.05 | 43.21 | — | — | 45.05 | — | |
| CoTBackbone=Llama3-8B2025.05 | 43.21 | — | — | 45.05 | — | |
| ATCBackbone=gpt-4o-mini, Paradigm=Code-based2025.02 | — | 70.21 | — | — | — | |
| Base RetrieverBase Model=Qwen3-30B2026.01 | — | — | — | — | 30.2 | |
| ChameleonBackbone=gpt-4o-mini, Paradigm=Text-based2025.02 | — | 74.87 | — | — | — | |
| CodeActBackbone=gpt-4o-mini, Paradigm=Code-based2025.02 | — | 75.94 | — | — | — | |
| ConAgentsBackbone=gpt-4o-mini, Paradigm=Text-based2025.02 | — | 67.26 | — | — | — | |
| EasyToolBackbone=gpt-4o-mini, Paradigm=Text-based2025.02 | — | 74.97 | — | — | — | |
| Hammer2-0.5BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 0.712 | 0.5903 | — | — | |
| Hammer2-1.5BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 0.8863 | 0.7926 | — | — | |
| Hammer2-3BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 0.8863 | 0.7904 | — | — | |
| Hammer2-7BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 0.8891 | 0.8128 | — | — | |
| Llama-3.1-70B (Base)Backbone=Llama-3.1-70B2026.04 | — | — | — | — | 54.89 | |
| Llama-3.1-8B (Base)Backbone=Llama-3.1-8B2026.04 | — | — | — | — | 62.91 | |
| Llama3-70BModel Series=Llama3 Series, Method=Baseline2025.05 | — | 0.8647 | 0.7887 | — | — | |
| Llama3-70BModel Series=Llama3 Series, Method=HiTEC-KTO2025.05 | — | 0.8688 | 0.7971 | — | — | |
| Llama3-70BModel Series=Llama3.1 Series, Method=HiTEC-KTO2025.05 | — | 0.9078 | 0.7707 | — | — | |
| Llama3-8BModel Series=Llama3 Series, Method=Baseline2025.05 | — | 0.6919 | 0.6773 | — | — | |
| Llama3-8BModel Series=Llama3 Series, Method=HiTEC-KTO2025.05 | — | 0.9499 | 0.8499 | — | — | |
| Llama3.1-70BModel Series=Llama3.1 Series, Method=Baseline2025.05 | — | 0.9015 | 0.7642 | — | — | |
| Llama3.1-8BModel Series=Llama3.1 Series, Method=Baseline2025.05 | — | 0.7306 | 0.6362 | — | — | |
| Llama3.1-8BModel Series=Llama3.1 Series, Method=HiTEC-KTO2025.05 | — | 0.8747 | 0.8099 | — | — | |
| Phi-4 (Base)Backbone=Phi-42026.04 | — | — | — | — | 34.09 | |
| Q2DBase Model=Qwen3-30B2026.01 | — | — | — | — | 31.8 | |
| Q2EBase Model=Qwen3-30B2026.01 | — | — | — | — | 31.3 | |
| Q2PBase Model=Qwen3-30B2026.01 | — | — | — | — | 30.1 | |
| Qwen2.5-0.5BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 0.8829 | 0.7834 | — | — | |
| Qwen2.5-1.5BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 0.8892 | 0.7946 | — | — | |
| Qwen2.5-3BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 0.8907 | 0.8016 | — | — | |
| Qwen2.5-7BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 0.8938 | 0.8167 | — | — | |
| Re-InvokeBase Model=Qwen3-30B2026.01 | — | — | — | — | 30.5 | |
| ReActBackbone=gpt-4o-mini, Paradigm=Text-based2025.02 | — | 73.93 | — | — | — | |
| RestGPTBackbone=gpt-4o-mini, Paradigm=Text-based2025.02 | — | 65.47 | — | — | — | |
| ToolCoderBackbone=gpt-4o-mini, Paradigm=Code-based2025.02 | — | 83.08 | — | — | — | |
| ToolCoder w/o Error ReflectionBackbone=gpt-4o-mini, Paradigm=Code-based, Error Reflection=false2025.02 | — | 79.85 | — | — | — | |
| ToolCoder w/o Reusable RepositoryBackbone=gpt-4o-mini, Paradigm=Code-based, Reusable Repository=false, Note=Mechanism disabled in this dataset2025.02 | — | 83.08 | — | — | — | |
| ToolDialBackbone=Llama-3.1-8B2026.04 | — | — | — | — | 59.15 | |
| ToolDialBackbone=Llama-3.1-70B2026.04 | — | — | — | — | 71.93 | |
| ToolDialBackbone=Phi-42026.04 | — | — | — | — | 55.64 | |
| ToolFlowBackbone=Llama-3.1-8B2026.04 | — | — | — | — | 67.92 | |
| ToolFlowBackbone=Llama-3.1-70B2026.04 | — | — | — | — | 70.68 | |
| ToolFlowBackbone=Phi-42026.04 | — | — | — | — | 49.12 | |
| TOOLQPBase Model=Qwen3-30B2026.01 | — | — | — | — | 33 | |
| ToolWeave (G)Backbone=Llama-3.1-8B2026.04 | — | — | — | — | 75.44 | |
| ToolWeave (G)Backbone=Llama-3.1-70B2026.04 | — | — | — | — | 71.43 | |
| ToolWeave (G)Backbone=Phi-42026.04 | — | — | — | — | 76.69 | |
| ToolWeave (O)Backbone=Llama-3.1-8B2026.04 | — | — | — | — | 68.92 | |
| ToolWeave (O)Backbone=Llama-3.1-70B2026.04 | — | — | — | — | 71.18 | |
| ToolWeave (O)Backbone=Phi-42026.04 | — | — | — | — | 71.68 | |
| ToolWeave-R (G)Backbone=Llama-3.1-8B2026.04 | — | — | — | — | 68.17 | |
| ToolWeave-R (G)Backbone=Llama-3.1-70B2026.04 | — | — | — | — | 71.93 | |
| ToolWeave-R (G)Backbone=Phi-42026.04 | — | — | — | — | 75.19 |