Tool Calling on API-Bank L-1 v1 (test)
90.78F1 ScoreHiTEC-KTO
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HiTEC-KTOModel Series=Llama3.1 Series, Model=Llama3-70B2025.05 | 90.78 | 77.07 | — | — | — | |
| BaselineModel Series=Llama3.1 Series, Model=Llama3.1-70B2025.05 | 90.15 | 76.42 | — | — | — | |
| HiTEC-KTOModel Series=Qwen2.5 Series, Model=Qwen2.5-7B2025.05 | 89.38 | 81.67 | — | — | — | |
| HiTEC-KTOModel Series=Qwen2.5 Series, Model=Qwen2.5-3B2025.05 | 89.07 | 80.16 | — | — | — | |
| HiTEC-KTOModel Series=Qwen2.5 Series, Model=Qwen2.5-1.5B2025.05 | 88.92 | 79.46 | — | — | — | |
| BaselineModel Series=Qwen2.5 Series, Model=Hammer2-7B2025.05 | 88.91 | 81.28 | — | — | — | |
| BaselineModel Series=Qwen2.5 Series, Model=Hammer2-1.5B2025.05 | 88.63 | 79.26 | — | — | — | |
| BaselineModel Series=Qwen2.5 Series, Model=Hammer2-3B2025.05 | 88.63 | 79.04 | — | — | — | |
| HiTEC-KTOModel Series=Qwen2.5 Series, Model=Qwen2.5-0.5B2025.05 | 88.29 | 78.34 | — | — | — | |
| HiTEC-KTOModel Series=Llama3.1 Series, Model=Llama3.1-8B2025.05 | 87.47 | 80.99 | — | — | — | |
| BaselineModel Series=Llama3.1 Series, Model=Llama3.1-8B2025.05 | 73.06 | 63.62 | — | — | — | |
| BaselineModel Series=Qwen2.5 Series, Model=Hammer2-0.5B2025.05 | 71.2 | 59.03 | — | — | — | |
| Hammer2.1-1.5BBase Model=Hammer2.1-1.5B2025.10 | — | — | 96.55 | 89.96 | 93.26 | |
| Hammer2.1-1.5B + Best of NBase Model=Hammer2.1-1.5B, Inference strategy=Best of N2025.10 | — | — | 97.29 | 90.25 | 93.77 | |
| Hammer2.1-1.5B + MajorityBase Model=Hammer2.1-1.5B, Inference strategy=Majority2025.10 | — | — | 93.93 | 85.71 | 89.82 | |
| Hammer2.1-1.5B + Token-level Beam SearchBase Model=Hammer2.1-1.5B, Inference strategy=Token-level Beam Search2025.10 | — | — | 91.41 | 84.78 | 88.1 | |
| Hammer2.1-1.5B + ToolPRMBase Model=Hammer2.1-1.5B, Inference strategy=ToolPRM2025.10 | — | — | 97.82 | 90.88 | 94.35 |