Video Counting on Molmo2-VC
37.1AccuracyGemini 3 Pro
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini 3 ProModel Category=API call only, Frames=384, Decoding=greedy2026.01 | 37.1 | 53.1 | |
| Gemini 3 ProModel category=API call only2026.03 | 37.1 | 53.1 | |
| MolmoPoint-Vid-8BModel category=MolmoPoint2026.03 | 36 | 58.7 | |
| GPT-5Model Category=API call only, Frames=384, Decoding=greedy2026.01 | 35.8 | 50.3 | |
| Gemini 2.5 ProModel Category=API call only, Frames=384, Decoding=greedy2026.01 | 35.8 | 56.5 | |
| GPT-5Model category=API call only2026.03 | 35.8 | 50.3 | |
| Gemini 2.5 ProModel category=API call only2026.03 | 35.8 | 56.5 | |
| MolmoPoint-8BModel category=MolmoPoint2026.03 | 35.6 | 54.6 | |
| Molmo2-8BModel Category=Molmo2 family: Open weights, Open data (no distillation), Open code, Frames=384, Decoding=greedy2026.01 | 35.5 | 53.3 | |
| Molmo2-8BModel category=Fully open2026.03 | 35.5 | 53.3 | |
| Molmo2-4BModel Category=Molmo2 family: Open weights, Open data (no distillation), Open code, Frames=384, Decoding=greedy2026.01 | 34.3 | 56.1 | |
| Molmo2-4BModel category=Fully open2026.03 | 34.3 | 56.1 | |
| Molmo2-O-7BModel Category=Molmo2 family: Open weights, Open data (no distillation), Open code, Frames=384, Decoding=greedy2026.01 | 33.2 | 50.5 | |
| Molmo2-O-7BModel category=Fully open2026.03 | 33.2 | 50.5 | |
| Gemini 2.5 FlashModel Category=API call only, Frames=384, Decoding=greedy2026.01 | 31.9 | 48.2 | |
| Gemini 2.5 FlashModel category=API call only2026.03 | 31.9 | 48.2 | |
| GPT-5 miniModel Category=API call only, Frames=384, Decoding=greedy2026.01 | 29.8 | 49.3 | |
| GPT-5 miniModel category=API call only2026.03 | 29.8 | 49.3 | |
| Qwen3-VL-8BModel Category=Open weights only, Frames=384, Decoding=greedy2026.01 | 29.6 | 47.7 | |
| Qwen3-VL-8BModel category=Open weights2026.03 | 29.6 | 47.7 | |
| Claude Sonnet 4.5Model Category=API call only, Frames=384, Decoding=greedy2026.01 | 27.2 | 45.1 | |
| Claude Sonnet 4.5Model category=API call only2026.03 | 27.2 | 45.1 | |
| Qwen3-VL-4BModel Category=Open weights only, Frames=384, Decoding=greedy2026.01 | 25.3 | 44.3 | |
| Qwen3-VL-4BModel category=Open weights2026.03 | 25.3 | 44.3 |