Multimodal tool-use on GTA
67.66Answer AccuracyReGRPO (default, λval = 0)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReGRPO (default, λval = 0)Controller=MAT-Qwen2-VL-7B2026.06 | 67.66 | 76.35 | 93.77 | |
| GPT-5Params=-2026.03 | 60.9 | 68.3 | 98.7 | |
| SPORT AgentController=Tuned-Qwen2-VL-7B2025.04 | 60.26 | 72.41 | 91.87 | |
| SPORT AgentController=Tuned-Qwen2-VL-7B2026.06 | 60.26 | 72.41 | 91.87 | |
| GPT-4.1Params=-2026.03 | 58.4 | 65.1 | 94.3 | |
| HF AgentController=GPT-4o-mini2025.04 | 57.69 | 56.1 | 100 | |
| HF AgentController=GPT-4o-mini2026.06 | 57.69 | 56.1 | 100 | |
| GPT-4oParams=-2026.03 | 57.1 | 63.4 | 95.1 | |
| HF AgentController=GPT-4o2025.04 | 57.05 | 63.41 | 95.12 | |
| HF AgentController=GPT-4o2026.06 | 57.05 | 63.41 | 95.12 | |
| TraceR1Params=8B2026.03 | 56.7 | 65.7 | 87.4 | |
| T3-AgentController=MAT-Qwen2-VL-7B2025.04 | 53.85 | 64.63 | 84.32 | |
| T3-AgentController=MAT-Qwen2-VL-7B2026.06 | 53.85 | 64.63 | 84.32 | |
| T3-AgentParams=7B2026.03 | 53.8 | 64.6 | 84.3 | |
| T3-AgentController=MAT-MiniCPM-V-8.5B2025.04 | 52.56 | 65.85 | 80.49 | |
| T3-AgentController=MAT-MiniCPM-V-8.5B2026.06 | 52.56 | 65.85 | 80.49 | |
| Qwen3-VL-8BParams=8B2026.03 | 49.2 | 56.8 | 74.2 | |
| Qwen2.5-VL-14BParams=14B2026.03 | 46.8 | 55.4 | 69.8 | |
| Lego AgentController=GPT-42025.04 | 46.59 | — | — | |
| Lego AgentController=GPT-42026.06 | 46.59 | — | — | |
| Qwen2.5-VL-7BParams=7B2026.03 | 44.2 | 50.6 | 69.1 | |
| HF AgentController=Qwen2-VL-7B2025.04 | 42.31 | 44.85 | 65.19 | |
| HF AgentController=Qwen2-VL-7B2026.06 | 42.31 | 44.85 | 65.19 | |
| Lego AgentController=GPT-4o2025.04 | 41.52 | — | — | |
| Lego AgentController=GPT-4o2026.06 | 41.52 | — | — | |
| HF AgentController=MiniCPM-V-8.5B2025.04 | 33.97 | 36.59 | 56.1 | |
| HF AgentController=MiniCPM-V-8.5B2026.06 | 33.97 | 36.59 | 56.1 | |
| HF AgentController=InternVL2-8B2025.04 | 32.05 | 36.75 | 52.18 | |
| HF AgentController=InternVL2-8B2026.06 | 32.05 | 36.75 | 52.18 | |
| DeepSeek-VL2Params=72B2026.03 | 23.2 | 49.4 | 57.2 | |
| LLAVA-NeXT-8BParams=8B2026.03 | 14.1 | 14.9 | 25.1 | |
| HF AgentController=LLaVA-NeXT-8B2025.04 | 14.1 | 14.97 | 25.08 | |
| HF AgentController=LLaVA-NeXT-8B2026.06 | 14.1 | 14.97 | 25.08 |