Tool Calling on Seal-Tools Single-Tool
98.14Name Match ScoreLlama3-70B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Llama3-70BModel Series=Llama3.1 Series, Method=HiTEC-KTO2025.05 | 98.14 | 90.01 | |
| HiTEC-KTOModel=Llama3.1-70B2025.05 | 98.14 | 90.01 | |
| HiTEC-ICLModel=GPT-3.5-Turbo-01252025.05 | 97.61 | 90.21 | |
| Llama3.1-70BModel Series=Llama3.1 Series, Method=Baseline2025.05 | 97.42 | 86.19 | |
| Qwen2.5-1.5BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | 96.99 | 89.75 | |
| HiTEC-ICLBackbone=Llama3.1-8B2025.05 | 96.95 | 87.94 | |
| Qwen2.5-0.5BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | 96.8 | 88.18 | |
| Llama3-70BModel Series=Llama3 Series, Method=HiTEC-KTO2025.05 | 96.47 | 88.37 | |
| Llama3-8BModel Series=Llama3 Series, Method=HiTEC-KTO2025.05 | 96.43 | 81.93 | |
| Qwen2.5-3BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | 96.28 | 89.51 | |
| Qwen2.5-7BModel Series=Qwen2.5 Series, Method=HiTEC-KTO2025.05 | 96.27 | 89.29 | |
| HiTEC-KTOModel=Qwen2.5-7B2025.05 | 96.27 | 89.29 | |
| Hammer2-1.5BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | 96.1 | 87.16 | |
| HiTEC-ICLBackbone=GPT-4-Turbo2025.05 | 96 | 89.37 | |
| HiTEC-ICLModel=Llama3.1-70B2025.05 | 95.86 | 87.19 | |
| Llama3.1-8BModel Series=Llama3.1 Series, Method=Baseline2025.05 | 94.97 | 86.53 | |
| VanillaBackbone=Llama3.1-8B2025.05 | 94.97 | 86.53 | |
| VanillaBackbone=GPT-4-Turbo2025.05 | 94.86 | 88.55 | |
| Hammer2-7BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | 94.62 | 87.84 | |
| Llama3.1-8BModel Series=Llama3.1 Series, Method=HiTEC-KTO2025.05 | 94.44 | 86.18 | |
| HiTEC-ICLBackbone=Llama3-8B2025.05 | 94.4 | 87.27 | |
| Hammer2-0.5BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | 93.86 | 83.03 | |
| Hammer2-3BModel Series=Qwen2.5 Series, Method=Baseline2025.05 | 93.08 | 85.6 | |
| CoTBackbone=GPT-4-Turbo2025.05 | 91.5 | 86.07 | |
| CoTBackbone=Llama3.1-8B2025.05 | 88.55 | 80.36 | |
| Llama3-70BModel Series=Llama3 Series, Method=Baseline2025.05 | 88 | 86.81 | |
| Llama3-8BModel Series=Llama3 Series, Method=Baseline2025.05 | 86.54 | 82.51 | |
| VanillaBackbone=Llama3-8B2025.05 | 86.54 | 82.51 | |
| Function CallingBackbone=GPT-4-Turbo2025.05 | 84.94 | 84.88 | |
| CoTBackbone=Llama3-8B2025.05 | 76.89 | 75.16 |