Task planning on MomaGraph-Bench Overall 1.0
73.9AccuracyClaude-4.5-Sonnet
Evaluation results
| Method | Links | |
|---|---|---|
| Claude-4.5-SonnetReasoning Protocol=w/ Graph2025.12 | 73.9 | |
| GPT-5Reasoning Protocol=w/ Graph2025.12 | 71.6 | |
| Gemini-2.5-ProReasoning Protocol=w/ Graph2025.12 | 71.6 | |
| MomaGraph-R1Params=7B, Reasoning Protocol=w/ Graph2025.12 | 71.6 | |
| Claude-4.5-SonnetReasoning Protocol=w/o Graph2025.12 | 69.8 | |
| Gemini-2.5-ProReasoning Protocol=w/o Graph2025.12 | 67.6 | |
| GPT-5Reasoning Protocol=w/o Graph2025.12 | 66.5 | |
| MomaGraph-R1Params=7B, Reasoning Protocol=w/o Graph2025.12 | 65.1 | |
| Qwen2.5-VL-7B-InstructParams=7B, Reasoning Protocol=w/ Graph2025.12 | 60.2 | |
| Qwen2.5-VL-7B-InstructParams=7B, Reasoning Protocol=w/o Graph2025.12 | 57.2 | |
| LLaVA-OnevisionParams=7B, Reasoning Protocol=w/ Graph2025.12 | 55.6 | |
| DeepSeek-VL2Params=4.5B, Reasoning Protocol=w/ Graph2025.12 | 54.3 | |
| LLaVA-OnevisionParams=7B, Reasoning Protocol=w/o Graph2025.12 | 53.5 | |
| DeepSeek-VL2Params=4.5B, Reasoning Protocol=w/o Graph2025.12 | 52 | |
| InternVL2.5Params=8B, Reasoning Protocol=w/ Graph2025.12 | 51.1 | |
| InternVL2.5Params=8B, Reasoning Protocol=w/o Graph2025.12 | 48.4 | |
| LLaVA-V1.5Params=7B, Reasoning Protocol=w/ Graph2025.12 | 44.8 | |
| LLaVA-V1.5Params=7B, Reasoning Protocol=w/o Graph2025.12 | 44.1 | |
| InstructBLIPParams=7B, Reasoning Protocol=w/ Graph2025.12 | 39.5 | |
| InstructBLIPParams=7B, Reasoning Protocol=w/o Graph2025.12 | 39 |