Task Planning on MomaGraph-Bench 1.0 (Tier 1)
83.7AccuracyClaude-4.5-Sonnet
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-4.5-SonnetModel Type=Closed Source, Reasoning Protocol=w/ Graph2025.12 | 83.7 | |
| GPT-5Model Type=Closed Source, Reasoning Protocol=w/ Graph2025.12 | 79.8 | |
| Gemini-2.5-ProModel Type=Closed Source, Reasoning Protocol=w/ Graph2025.12 | 79 | |
| Claude-4.5-SonnetModel Type=Closed Source, Reasoning Protocol=w/o Graph2025.12 | 77.3 | |
| GPT-5Model Type=Closed Source, Reasoning Protocol=w/o Graph2025.12 | 77.3 | |
| Gemini-2.5-ProModel Type=Closed Source, Reasoning Protocol=w/o Graph2025.12 | 76.6 | |
| MomaGraph-R1Params=7B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 76.4 | |
| MomaGraph-R1Params=7B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 70.2 | |
| Qwen2.5-VL-7B-InstructParams=7B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 66.3 | |
| LLaVA-OnevisionParams=7B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 63.8 | |
| Qwen2.5-VL-7B-InstructParams=7B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 62.1 | |
| LLaVA-OnevisionParams=7B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 60 | |
| DeepSeek-VL2Params=4.5B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 56.9 | |
| DeepSeek-VL2Params=4.5B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 54.2 | |
| InternVL2.5Params=8B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 53.6 | |
| LLaVA-V1.5Params=7B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 53.4 | |
| LLaVA-V1.5Params=7B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 51 | |
| InternVL2.5Params=8B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 51 | |
| InstructBLIPParams=7B, Model Type=Open Source, Reasoning Protocol=w/ Graph2025.12 | 44.1 | |
| InstructBLIPParams=7B, Model Type=Open Source, Reasoning Protocol=w/o Graph2025.12 | 43.1 |