Multimodal Agent Task on VisualToolBench
46.5Average@4XSkill
Evaluation results
| Method | Links | ||
|---|---|---|---|
| XSkillModel=Gemini-3-Flash2026.03 | 46.5 | 64.02 | |
| w/ ToolsModel=Gemini-3-Flash2026.03 | 41.94 | 60.75 | |
| AWMModel=Gemini-3-Flash2026.03 | 41.94 | 59.35 | |
| Agent-KBModel=Gemini-3-Flash2026.03 | 41.75 | 61.21 | |
| DCModel=Gemini-3-Flash2026.03 | 41.7 | 59.81 | |
| XSkillModel=Gemini-2.5-Pro2026.03 | 30.49 | 46.73 | |
| Qwen3.5-PlusEvaluation Protocol=Vanilla Tools2026.05 | 29.21 | 49.07 | |
| Agent-KBModel=Gemini-2.5-Pro2026.03 | 26.75 | 41.12 | |
| Qwen3.5-35B-A3BEvaluation Protocol=w/ ACE-SKILL2026.05 | 26.4 | 41.59 | |
| AWMModel=Gemini-2.5-Pro2026.03 | 25.93 | 39.25 | |
| w/ ToolsModel=Gemini-2.5-Pro2026.03 | 25.35 | 40.65 | |
| Gemini-2.5-ProEvaluation Protocol=Vanilla Tools2026.05 | 25.35 | 40.65 | |
| No ToolsModel=Gemini-3-Flash2026.03 | 25.12 | 36.92 | |
| XSkillModel=o4-mini2026.03 | 25 | 41.12 | |
| DCModel=Gemini-2.5-Pro2026.03 | 24.77 | 37.38 | |
| Agent-KBModel=GPT-5-mini2026.03 | 24.77 | 38.32 | |
| XSkillModel=GPT-5-mini2026.03 | 24.53 | 37.85 | |
| w/ ToolsModel=GPT-5-mini2026.03 | 24.3 | 37.85 | |
| GPT-5-miniEvaluation Protocol=Vanilla Tools2026.05 | 24.3 | 37.85 | |
| DCModel=GPT-5-mini2026.03 | 23.83 | 35.05 | |
| AWMModel=GPT-5-mini2026.03 | 23.25 | 34.58 | |
| Agent-KBModel=o4-mini2026.03 | 22.78 | 36.92 | |
| Qwen3.5-35B-A3BEvaluation Protocol=w/ XSkill2026.05 | 22.43 | 35.51 | |
| Qwen3.5-35B-A3BEvaluation Protocol=Vanilla Tools2026.05 | 21.38 | 39.72 | |
| AWMModel=o4-mini2026.03 | 21.14 | 36.92 | |
| Kimi-K2.5Evaluation Protocol=Vanilla Tools2026.05 | 21.03 | 34.11 | |
| No ToolsModel=Gemini-2.5-Pro2026.03 | 20.91 | 28.97 | |
| Gemini-2.5-ProEvaluation Protocol=Native Direct Answer2026.05 | 20.91 | 28.97 | |
| DCModel=o4-mini2026.03 | 20.44 | 33.64 | |
| w/ ToolsModel=o4-mini2026.03 | 19.63 | 34.11 | |
| o4-miniEvaluation Protocol=Vanilla Tools2026.05 | 19.63 | 34.11 | |
| GLM-4.6VEvaluation Protocol=Native Direct Answer2026.05 | 17.76 | 34.11 | |
| Qwen3.5-PlusEvaluation Protocol=Native Direct Answer2026.05 | 17.17 | 28.04 | |
| GLM-4.6VEvaluation Protocol=Vanilla Tools2026.05 | 15.89 | 31.78 | |
| Qwen3.5-35B-A3BEvaluation Protocol=Native Direct Answer2026.05 | 15.65 | 28.04 | |
| No ToolsModel=o4-mini2026.03 | 14.72 | 27.57 | |
| o4-miniEvaluation Protocol=Native Direct Answer2026.05 | 14.72 | 27.57 | |
| No ToolsModel=GPT-5-mini2026.03 | 13.9 | 22.9 | |
| GPT-5-miniEvaluation Protocol=Native Direct Answer2026.05 | 13.9 | 22.9 | |
| Kimi-K2.5Evaluation Protocol=Native Direct Answer2026.05 | 12.85 | 19.63 |