Multimodal Tool-Use on TIR-Bench
50Avg@4Qwen3.5-35B-A3B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3.5-35B-A3BEvaluation Protocol=w/ ACE-SKILL2026.05 | 50 | 71.5 | |
| Qwen3.5-35B-A3BEvaluation Protocol=w/ XSkill2026.05 | 40.62 | 68 | |
| Qwen3.5-PlusEvaluation Protocol=Vanilla Tools2026.05 | 40.25 | 62.5 | |
| Qwen3.5-35B-A3BEvaluation Protocol=Vanilla Tools2026.05 | 39.37 | 65 | |
| Qwen3.5-PlusEvaluation Protocol=Native Direct Answer2026.05 | 29.88 | 47.5 | |
| Gemini-2.5-ProEvaluation Protocol=Vanilla Tools2026.05 | 28.38 | 54 | |
| o4-miniEvaluation Protocol=Vanilla Tools2026.05 | 24.62 | 49.5 | |
| GPT-5-miniEvaluation Protocol=Vanilla Tools2026.05 | 23.5 | 50.5 | |
| Qwen3.5-35B-A3BEvaluation Protocol=Native Direct Answer2026.05 | 23 | 43 | |
| Gemini-2.5-ProEvaluation Protocol=Native Direct Answer2026.05 | 21.37 | 40 | |
| o4-miniEvaluation Protocol=Native Direct Answer2026.05 | 20.13 | 45 | |
| GPT-5-miniEvaluation Protocol=Native Direct Answer2026.05 | 20 | 46.5 | |
| Kimi-K2.5Evaluation Protocol=Vanilla Tools2026.05 | 19.38 | 37 | |
| GLM-4.6VEvaluation Protocol=Vanilla Tools2026.05 | 18.75 | 42 | |
| GLM-4.6VEvaluation Protocol=Native Direct Answer2026.05 | 17.5 | 28 | |
| Kimi-K2.5Evaluation Protocol=Native Direct Answer2026.05 | 17.37 | 28 |