Social Video Reasoning on VAGUE (test)
77.4AccuracyGemini-3.0-Flash
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-3.0-FlashStrategy=CoCoT, Model Category=Reasoning VLMs, Thinking mode=true2025.07 | 77.4 | |
| Gemini-3.0-FlashStrategy=Direct, Model Category=Reasoning VLMs, Thinking mode=true2025.07 | 73.05 | |
| Gemini-3.0-FlashStrategy=CoT, Model Category=Reasoning VLMs, Thinking mode=true2025.07 | 72.27 | |
| Gemini-2.5-ProStrategy=CoCoT, Model Category=Standard VLMs2025.07 | 67.62 | |
| GPT-4oStrategy=CoCoT, Model Category=Standard VLMs2025.07 | 67.43 | |
| Claude-3.5-SonnetStrategy=CoCoT, Model Category=Standard VLMs2025.07 | 67.3 | |
| Claude-3.5-SonnetStrategy=Direct, Model Category=Standard VLMs2025.07 | 62.92 | |
| OpenAI o1-fullStrategy=CoT, Model Category=Reasoning VLMs2025.07 | 62.57 | |
| GPT-5.2-HighStrategy=CoCoT, Model Category=Reasoning VLMs2025.07 | 62.02 | |
| GPT-4oStrategy=Direct, Model Category=Standard VLMs2025.07 | 61.6 | |
| OpenAI o1-fullStrategy=CoCoT, Model Category=Reasoning VLMs2025.07 | 61.59 | |
| GPT-4oStrategy=CoT, Model Category=Standard VLMs2025.07 | 60 | |
| Gemini-3.0-FlashStrategy=CCoT, Model Category=Reasoning VLMs, Thinking mode=true2025.07 | 60 | |
| GPT-5.2-HighStrategy=CoT, Model Category=Reasoning VLMs2025.07 | 59.99 | |
| OpenAI o1-fullStrategy=Direct, Model Category=Reasoning VLMs2025.07 | 59.82 | |
| Claude-3.5-SonnetStrategy=CoT, Model Category=Standard VLMs2025.07 | 59.52 | |
| GPT-5.2-HighStrategy=Direct, Model Category=Reasoning VLMs2025.07 | 59.21 | |
| Gemini-2.5-ProStrategy=CoT, Model Category=Standard VLMs2025.07 | 58.32 | |
| LLaVA-OneVision-7BStrategy=CoCoT, Model Category=Open-Source VLMs2025.07 | 54.65 | |
| LLaVA-OneVision-7BStrategy=CoT, Model Category=Open-Source VLMs2025.07 | 53.31 | |
| Gemini-2.5-ProStrategy=Direct, Model Category=Standard VLMs2025.07 | 53.25 | |
| Claude-3.5-SonnetStrategy=CCoT, Model Category=Standard VLMs2025.07 | 52.02 | |
| Idefics3-8BStrategy=Direct, Model Category=Open-Source VLMs2025.07 | 50.98 | |
| LLaVA-v1.6-7BStrategy=CoCoT, Model Category=Open-Source VLMs2025.07 | 50.21 | |
| GPT-4oStrategy=CCoT, Model Category=Standard VLMs2025.07 | 50.12 | |
| Molmo-7BStrategy=CoCoT, Model Category=Open-Source VLMs2025.07 | 50 | |
| LLaVA-OneVision-7BStrategy=Direct, Model Category=Open-Source VLMs2025.07 | 49.73 | |
| OpenAI o1-fullStrategy=CCoT, Model Category=Reasoning VLMs2025.07 | 48.84 | |
| Idefics3-8BStrategy=CoT, Model Category=Open-Source VLMs2025.07 | 48.54 | |
| Idefics3-8BStrategy=CoCoT, Model Category=Open-Source VLMs2025.07 | 48.24 | |
| Molmo-7BStrategy=Direct, Model Category=Open-Source VLMs2025.07 | 48 | |
| LLaVA-v1.6-7BStrategy=CoT, Model Category=Open-Source VLMs2025.07 | 47.44 | |
| GPT-5.2-HighStrategy=CCoT, Model Category=Reasoning VLMs2025.07 | 47.31 | |
| Gemini-2.5-ProStrategy=CCoT, Model Category=Standard VLMs2025.07 | 46.7 | |
| Molmo-7BStrategy=CoT, Model Category=Open-Source VLMs2025.07 | 46 | |
| LLaVA-v1.6-7BStrategy=Direct, Model Category=Open-Source VLMs2025.07 | 46 | |
| Qwen2-VL-7BStrategy=CoCoT, Model Category=Open-Source VLMs2025.07 | 44.42 | |
| LLaVA-OneVision-7BStrategy=CCoT, Model Category=Open-Source VLMs2025.07 | 44.3 | |
| OpenAI o3-miniStrategy=CoCoT, Model Category=Reasoning VLMs2025.07 | 43.64 | |
| Idefics3-8BStrategy=CCoT, Model Category=Open-Source VLMs2025.07 | 42.43 | |
| OpenAI o3-miniStrategy=CoT, Model Category=Reasoning VLMs2025.07 | 40.21 | |
| Molmo-7BStrategy=CCoT, Model Category=Open-Source VLMs2025.07 | 40.19 | |
| LLaVA-v1.6-7BStrategy=CCoT, Model Category=Open-Source VLMs2025.07 | 40.1 | |
| Qwen2-VL-7BStrategy=Direct, Model Category=Open-Source VLMs2025.07 | 38.52 | |
| Qwen2-VL-7BStrategy=CoT, Model Category=Open-Source VLMs2025.07 | 38.49 | |
| OpenAI o3-miniStrategy=Direct, Model Category=Reasoning VLMs2025.07 | 38.15 | |
| OpenAI o3-miniStrategy=CCoT, Model Category=Reasoning VLMs2025.07 | 34.43 | |
| Qwen2-VL-7BStrategy=CCoT, Model Category=Open-Source VLMs2025.07 | 34.02 |