Tool-use Factuality Evaluation on BFCL Task
76Factuality ScoreGPT-OSS-120B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-OSS-120BExperimental Setup=Optimized2026.05 | 76 | 129 | |
| Qwen-3-30B-ITExperimental Setup=Optimized2026.05 | 74 | 148 | |
| GPT-OSS-120BExperimental Setup=Always Tool2026.05 | 73 | 328 | |
| Gemma3-27B-ITExperimental Setup=Optimized2026.05 | 73 | 103 | |
| Qwen-3-30B-ITExperimental Setup=Always Tool2026.05 | 71 | 328 | |
| Mistral3.1-24B-ITExperimental Setup=Optimized2026.05 | 71 | 135 | |
| Qwen-3-30B-ITExperimental Setup=Self-decided2026.05 | 67 | 351 | |
| GPT-OSS-120BExperimental Setup=Self-decided2026.05 | 66 | 196 | |
| Gemma3-27B-ITExperimental Setup=Always Tool2026.05 | 66 | 328 | |
| Mistral3.1-24B-ITExperimental Setup=Always Tool2026.05 | 66 | 328 | |
| Llama3.2-3B-ITExperimental Setup=Optimized2026.05 | 66 | 160 | |
| GPT-OSS-120BExperimental Setup=LNE2026.05 | 65 | 207 | |
| Qwen3-30B-A3BExperimental Setup=Optimized2026.05 | 64 | 146 | |
| Llama3.2-3B-ITExperimental Setup=Always Tool2026.05 | 63 | 328 | |
| Gemma3-27B-ITExperimental Setup=Self-decided2026.05 | 62 | 360 | |
| Gemma3-27B-ITExperimental Setup=LNE2026.05 | 61 | 218 | |
| Qwen3-30B-A3BExperimental Setup=Always Tool2026.05 | 60 | 328 | |
| Mistral3.1-24B-ITExperimental Setup=Self-decided2026.05 | 60 | 302 | |
| Qwen-3-30B-ITExperimental Setup=LNE2026.05 | 58 | 206 | |
| Llama3.2-3B-ITExperimental Setup=Self-decided2026.05 | 58 | 346 | |
| Llama3.2-3B-ITExperimental Setup=LNE2026.05 | 58 | 263 | |
| Mistral3.1-24B-ITExperimental Setup=LNE2026.05 | 57 | 240 | |
| Qwen3-30B-A3BExperimental Setup=LNE2026.05 | 56 | 264 | |
| GPT-OSS-120BExperimental Setup=LUEx2026.05 | 55 | 130 | |
| Qwen3-30B-A3BExperimental Setup=Self-decided2026.05 | 52 | 261 | |
| Qwen-3-30B-ITExperimental Setup=LUEx2026.05 | 51 | 138 | |
| Gemma3-27B-ITExperimental Setup=LUEx2026.05 | 50 | 83 | |
| Mistral3.1-24B-ITExperimental Setup=LUEx2026.05 | 46 | 143 | |
| Qwen3-30B-A3BExperimental Setup=LUEx2026.05 | 42 | 156 | |
| Gemma3-27B-ITExperimental Setup=No Tool2026.05 | 42 | 0 | |
| Llama3.2-3B-ITExperimental Setup=LUEx2026.05 | 42 | 153 | |
| GPT-OSS-120BExperimental Setup=No Tool2026.05 | 36 | 0 | |
| Mistral3.1-24B-ITExperimental Setup=No Tool2026.05 | 30 | 0 | |
| Qwen-3-30B-ITExperimental Setup=No Tool2026.05 | 29 | 0 | |
| Qwen3-30B-A3BExperimental Setup=No Tool2026.05 | 20 | 0 | |
| Llama3.2-3B-ITExperimental Setup=No Tool2026.05 | 17 | 0 | |
| Llama3.2-3B-ITExperimental Setup=LUEx,d2026.05 | 16 | 161 | |
| Qwen3-30B-A3BExperimental Setup=LUEx,d2026.05 | 15 | 154 | |
| Qwen-3-30B-ITExperimental Setup=LUEx,d2026.05 | 13 | 134 | |
| Mistral3.1-24B-ITExperimental Setup=LUEx,d2026.05 | 13 | 132 | |
| GPT-OSS-120BExperimental Setup=LUEx,d2026.05 | 12 | 123 | |
| Gemma3-27B-ITExperimental Setup=LUEx,d2026.05 | 7 | 67 |