Task Planning on MomaGraph-Bench Tier 4 1.0
69.5AccuracyClaude-4.5-Sonnet
Evaluation results
| Method | Links | |
|---|---|---|
| Claude-4.5-SonnetReasoning Protocol=w/ Graph2025.12 | 69.5 | |
| MomaGraph-R1Params=7B, Reasoning Protocol=w/ Graph2025.12 | 68.1 | |
| Claude-4.5-SonnetReasoning Protocol=w/o Graph2025.12 | 65.2 | |
| Gemini-2.5-ProReasoning Protocol=w/ Graph2025.12 | 65.2 | |
| GPT-5Reasoning Protocol=w/ Graph2025.12 | 63.6 | |
| Gemini-2.5-ProReasoning Protocol=w/o Graph2025.12 | 60.8 | |
| MomaGraph-R1Params=7B, Reasoning Protocol=w/o Graph2025.12 | 60.8 | |
| Qwen2.5-VL-7B-InstructParams=7B, Reasoning Protocol=w/ Graph2025.12 | 59 | |
| Qwen2.5-VL-7B-InstructParams=7B, Reasoning Protocol=w/o Graph2025.12 | 56.5 | |
| GPT-5Reasoning Protocol=w/o Graph2025.12 | 54.5 | |
| DeepSeek-VL2Params=4.5B, Reasoning Protocol=w/ Graph2025.12 | 45.4 | |
| LLaVA-OnevisionParams=7B, Reasoning Protocol=w/o Graph2025.12 | 43.4 | |
| LLaVA-OnevisionParams=7B, Reasoning Protocol=w/ Graph2025.12 | 43.4 | |
| LLaVA-V1.5Params=7B, Reasoning Protocol=w/ Graph2025.12 | 40.9 | |
| DeepSeek-VL2Params=4.5B, Reasoning Protocol=w/o Graph2025.12 | 40.9 | |
| InternVL2.5Params=8B, Reasoning Protocol=w/ Graph2025.12 | 40.9 | |
| LLaVA-V1.5Params=7B, Reasoning Protocol=w/o Graph2025.12 | 38.9 | |
| InstructBLIPParams=7B, Reasoning Protocol=w/ Graph2025.12 | 36.3 | |
| InternVL2.5Params=8B, Reasoning Protocol=w/o Graph2025.12 | 33.3 | |
| InstructBLIPParams=7B, Reasoning Protocol=w/o Graph2025.12 | 31.8 |