Multi-modal Reasoning on MoMentS
72.26AccuracyGPT-4o
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4oStrategy=COCOT, Category=Standard VLMs2025.07 | 72.26 | |
| Gemini-3.0-Flash*Strategy=COCOT, Category=Reasoning VLMs, Thinking mode=true2025.07 | 71.31 | |
| GPT-4oStrategy=CCoT, Category=Standard VLMs2025.07 | 70.79 | |
| GPT-4oStrategy=Direct, Category=Standard VLMs2025.07 | 70.68 | |
| OpenAI o1-fullStrategy=COCOT, Category=Reasoning VLMs2025.07 | 69.15 | |
| Gemini-3.0-Flash*Strategy=CCoT, Category=Reasoning VLMs, Thinking mode=true2025.07 | 68.5 | |
| GPT-4oStrategy=CoT, Category=Standard VLMs2025.07 | 68.12 | |
| OpenAI o1-fullStrategy=Direct, Category=Reasoning VLMs2025.07 | 67.6 | |
| Gemini-3.0-Flash*Strategy=CoT, Category=Reasoning VLMs, Thinking mode=true2025.07 | 66.8 | |
| OpenAI o1-fullStrategy=CCoT, Category=Reasoning VLMs2025.07 | 66.5 | |
| Gemini-3.0-Flash*Strategy=Direct, Category=Reasoning VLMs, Thinking mode=true2025.07 | 64.8 | |
| OpenAI o1-fullStrategy=CoT, Category=Reasoning VLMs2025.07 | 64.19 | |
| Claude-3.5-SonnetStrategy=Direct, Category=Standard VLMs2025.07 | 63.75 | |
| Claude-3.5-SonnetStrategy=COCOT, Category=Standard VLMs2025.07 | 62.95 | |
| GPT-5.2-HighStrategy=Direct, Category=Reasoning VLMs2025.07 | 62.95 | |
| GPT-5.2-HighStrategy=COCOT, Category=Reasoning VLMs2025.07 | 62.55 | |
| GPT-5.2-HighStrategy=CoT, Category=Reasoning VLMs2025.07 | 62.43 | |
| GPT-5.2-HighStrategy=CCoT, Category=Reasoning VLMs2025.07 | 62.32 | |
| Claude-3.5-SonnetStrategy=CCoT, Category=Standard VLMs2025.07 | 62 | |
| Claude-3.5-SonnetStrategy=CoT, Category=Standard VLMs2025.07 | 61.35 | |
| Gemini-2.5-ProStrategy=COCOT, Category=Standard VLMs2025.07 | 60 | |
| OpenAI o3-miniStrategy=COCOT, Category=Reasoning VLMs2025.07 | 58 | |
| OpenAI o3-miniStrategy=Direct, Category=Reasoning VLMs2025.07 | 56 | |
| Gemini-2.5-ProStrategy=CCoT, Category=Standard VLMs2025.07 | 55.5 | |
| OpenAI o3-miniStrategy=CCoT, Category=Reasoning VLMs2025.07 | 55.5 | |
| Gemini-2.5-ProStrategy=Direct, Category=Standard VLMs2025.07 | 55 | |
| Gemini-2.5-ProStrategy=CoT, Category=Standard VLMs2025.07 | 53 | |
| OpenAI o3-miniStrategy=CoT, Category=Reasoning VLMs2025.07 | 52.94 | |
| LLaVA-v1.6-7BStrategy=Direct, Category=Open-Source VLMs2025.07 | 50.2 | |
| Qwen2-VL-7BStrategy=Direct, Category=Open-Source VLMs2025.07 | 49 | |
| LLaVA-OneVision-7BStrategy=CoT, Category=Open-Source VLMs2025.07 | 48.21 | |
| LLaVA-v1.6-7BStrategy=COCOT, Category=Open-Source VLMs2025.07 | 48 | |
| LLaVA-OneVision-7BStrategy=COCOT, Category=Open-Source VLMs2025.07 | 46.22 | |
| LLaVA-v1.6-7BStrategy=CCoT, Category=Open-Source VLMs2025.07 | 46 | |
| LLaVA-OneVision-7BStrategy=Direct, Category=Open-Source VLMs2025.07 | 45.82 | |
| LLaVA-OneVision-7BStrategy=CCoT, Category=Open-Source VLMs2025.07 | 45.5 | |
| Molmo-7BStrategy=COCOT, Category=Open-Source VLMs2025.07 | 45.23 | |
| Molmo-7BStrategy=CCoT, Category=Open-Source VLMs2025.07 | 44.03 | |
| Molmo-7BStrategy=Direct, Category=Open-Source VLMs2025.07 | 44 | |
| LLaVA-v1.6-7BStrategy=CoT, Category=Open-Source VLMs2025.07 | 44 | |
| Molmo-7BStrategy=CoT, Category=Open-Source VLMs2025.07 | 42 | |
| Qwen2-VL-7BStrategy=COCOT, Category=Open-Source VLMs2025.07 | 41.2 | |
| Qwen2-VL-7BStrategy=CCoT, Category=Open-Source VLMs2025.07 | 40.87 | |
| Idefics3-8BStrategy=Direct, Category=Open-Source VLMs2025.07 | 40 | |
| Idefics3-8BStrategy=COCOT, Category=Open-Source VLMs2025.07 | 37.51 | |
| Qwen2-VL-7BStrategy=CoT, Category=Open-Source VLMs2025.07 | 37.09 | |
| Idefics3-8BStrategy=CCoT, Category=Open-Source VLMs2025.07 | 36.73 | |
| Idefics3-8BStrategy=CoT, Category=Open-Source VLMs2025.07 | 35.1 |