Multimodal Physical Reasoning on PACC (test)
2.286Average ScoreGemini 2.5 Flash
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 FlashModel Access Type=Proprietary2026.05 | 2.286 | 2.469 | 2.441 | 2.833 | 2.3 | 2.571 | 1.875 | 1.818 | 2.2 | |
| GPT-4oModel Access Type=Proprietary2026.05 | 2.12 | 2.156 | 2.324 | 2.667 | 2.5 | 2.048 | 1.679 | 2.25 | 2.1 | |
| PhyARModel Access Type=Open-Source2026.05 | 2.036 | 2.375 | 2.294 | 1.611 | 2.05 | 1.619 | 1.732 | 2.523 | 2.95 | |
| Qwen3.5-35BModel Access Type=Open-Source, Model Parameters=35B2026.05 | 1.477 | 1.125 | 1.529 | 1.5 | 1.95 | 1.524 | 1.429 | 1.409 | 1.55 | |
| VideoLLaMA3-7BModel Access Type=Open-Source, Model Parameters=7B2026.05 | 1.237 | 1 | 1.118 | 1.222 | 1.2 | 1.262 | 1.268 | 1.386 | 1.35 | |
| MiniCPM-V 2.6Model Access Type=Open-Source2026.05 | 1.182 | 1.125 | 1.118 | 1.333 | 1.45 | 1.167 | 1.161 | 1.159 | 1.15 | |
| VideoLLaMA2-7BModel Access Type=Open-Source, Model Parameters=7B2026.05 | 1.159 | 1 | 1.118 | 1.111 | 1.25 | 1.238 | 1.071 | 1.204 | 1.25 | |
| InternVL2.5-8BModel Access Type=Open-Source, Model Parameters=8B2026.05 | 1.117 | 1 | 1.147 | 1 | 1.3 | 1.131 | 1.143 | 1.091 | 1.1 | |
| Flash-VStream-7BModel Access Type=Open-Source, Model Parameters=7B2026.05 | 1.101 | 1.125 | 1.176 | 1.222 | 1 | 1.024 | 1.107 | 1.182 | 1.05 | |
| Video-LLaVA-7BModel Access Type=Open-Source, Model Parameters=7B2026.05 | 1.081 | 1 | 1 | 1 | 1 | 1.095 | 1.143 | 1.136 | 1.15 | |
| Video-ChatGPT-7BModel Access Type=Open-Source, Model Parameters=7B2026.05 | 1.075 | 1 | 1.206 | 1.056 | 1.1 | 1.048 | 1.089 | 1.045 | 1.1 |