Tool Calling on API-Bank L-2
90.42Name Match F1HiTEC-KTO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| HiTEC-KTOModel=Llama3-8B2025.05 | 90.42 | — | — | 64.2 | |
| + Function CallingModel=GPT-3.5-Turbo-01252025.05 | 88.72 | — | — | 65.87 | |
| HiTEC-KTOModel=Qwen2.5-7B2025.05 | 88 | — | — | 59.93 | |
| HiTEC-KTOModel=Llama3.1-8B2025.05 | 85.61 | — | — | 61.02 | |
| HiTEC-ICLModel=GPT-3.5-Turbo-01252025.05 | 83.8 | — | — | 64.72 | |
| HiTEC-ICLModel=Llama3.1-8B2025.05 | 82.23 | — | — | 51.29 | |
| HiTEC-ICLBackbone=Llama3.1-8B2025.05 | 82.23 | — | — | 51.29 | |
| VanillaModel=GPT-3.5-Turbo-01252025.05 | 78.87 | — | — | 61.02 | |
| + Function CallingModel=GPT-4-Turbo2025.05 | 78.49 | — | — | 62.32 | |
| Function CallingBackbone=GPT-4-Turbo2025.05 | 78.49 | — | — | 62.32 | |
| VanillaModel=Llama3.1-8B2025.05 | 78.23 | — | — | 50.25 | |
| VanillaBackbone=Llama3.1-8B2025.05 | 78.23 | — | — | 50.25 | |
| HiTEC-ICLModel=Llama3-8B2025.05 | 77.3 | — | — | 56.01 | |
| HiTEC-ICLBackbone=Llama3-8B2025.05 | 77.3 | — | — | 56.01 | |
| VanillaModel=Llama3-8B2025.05 | 75.54 | — | — | 48.91 | |
| VanillaBackbone=Llama3-8B2025.05 | 75.54 | — | — | 48.91 | |
| CoTBackbone=Llama3.1-8B2025.05 | 75.54 | — | — | 48.52 | |
| HiTEC-ICLModel=GPT-4-Turbo2025.05 | 71.21 | — | — | 72.68 | |
| HiTEC-ICLBackbone=GPT-4-Turbo2025.05 | 71.21 | — | — | 72.68 | |
| VanillaModel=GPT-4-Turbo2025.05 | 71.09 | — | — | 70.43 | |
| VanillaBackbone=GPT-4-Turbo2025.05 | 71.09 | — | — | 70.43 | |
| + CoTModel=GPT-4-Turbo2025.05 | 67.86 | — | — | 65.74 | |
| CoTBackbone=GPT-4-Turbo2025.05 | 67.86 | — | — | 65.74 | |
| + CoTModel=GPT-3.5-Turbo-01252025.05 | 59.2 | — | — | 55.53 | |
| CoTBackbone=Llama3-8B2025.05 | 52.31 | — | — | 45.65 | |
| Base RetrieverBase Model=Qwen3-30B2026.01 | — | 57.8 | — | — | |
| Hammer2-0.5BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 43.32 | 38.22 | — | |
| Hammer2-1.5BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 80.51 | 62.82 | — | |
| Hammer2-3BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 77.11 | 57.58 | — | |
| Hammer2-7BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | — | 75.96 | 58.36 | — | |
| Llama3-70BModel Series=Llama3 Series, Method=Baseline2025.05 | — | 66.67 | 63.65 | — | |
| Llama3-70BModel Series=Llama3 Series, Method=HiTEC-KTO2025.05 | — | 75.61 | 63.08 | — | |
| Llama3-70BModel Series=Llama3.1 Series, Method=HiTEC-KTO2025.05 | — | 86.44 | 65.14 | — | |
| Llama3-8BModel Series=Llama3 Series, Method=Baseline2025.05 | — | 75.54 | 48.91 | — | |
| Llama3-8BModel Series=Llama3 Series, Method=HiTEC-KTO2025.05 | — | 90.42 | 64.2 | — | |
| Llama3.1-70BModel Series=Llama3.1 Series, Method=Baseline2025.05 | — | 80.34 | 62.2 | — | |
| Llama3.1-8BModel Series=Llama3.1 Series, Method=Baseline2025.05 | — | 78.23 | 50.25 | — | |
| Llama3.1-8BModel Series=Llama3.1 Series, Method=HiTEC-KTO2025.05 | — | 85.61 | 61.02 | — | |
| Q2DBase Model=Qwen3-30B2026.01 | — | 58.5 | — | — | |
| Q2EBase Model=Qwen3-30B2026.01 | — | 59.5 | — | — | |
| Q2PBase Model=Qwen3-30B2026.01 | — | 56.1 | — | — | |
| Qwen2.5-0.5BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 81.76 | 54.52 | — | |
| Qwen2.5-1.5BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 81.36 | 60.66 | — | |
| Qwen2.5-3BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 85.51 | 54.61 | — | |
| Qwen2.5-7BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | — | 88 | 59.93 | — | |
| Re-InvokeBase Model=Qwen3-30B2026.01 | — | 60.5 | — | — | |
| TOOLQPBase Model=Qwen3-30B2026.01 | — | 60.7 | — | — |