Physical Reasoning on PACC (test)
56.2CoherenceGemini 2.5 Flash
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 FlashModel Category=Proprietary, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 56.2 | 52.9 | 55.6 | 40 | 45.2 | 25 | 27.2 | 40 | 40.91 | |
| PhyARModel Category=Open-Source, Backbone=VideoLLaMA3-7B, Evaluation Protocol=fine-tuned, Frame Sampling=16 frames, Training Strategy=LoRA, Epochs=32026.05 | 50 | 52.9 | 22.2 | 50 | 28.6 | 21.4 | 68.2 | 70 | 41.56 | |
| GPT-4oModel Category=Proprietary, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 43.8 | 41.2 | 55.6 | 50 | 35.7 | 21.4 | 45.5 | 40 | 38.31 | |
| Qwen3.5-35BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 6.2 | 17.6 | 11.1 | 30 | 16.7 | 10.7 | 9.1 | 10 | 13.64 | |
| Flash-VStream-7BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 0 | 0 | 0 | 0 | 0 | 4.5 | 0 | 0.65 | |
| Video-ChatGPT-7BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 5.9 | 0 | 0 | 0 | 0 | 0 | 0 | 0.65 | |
| InternVL2.5-8BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 5.9 | 0 | 10 | 4.8 | 0 | 0 | 0 | 2.6 | |
| Video-LLaVA-7BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 0 | 0 | 0 | 4.8 | 7.1 | 0 | 0 | 2.6 | |
| MiniCPM-V 2.6Model Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 5.9 | 0 | 10 | 4.8 | 3.6 | 4.5 | 0 | 3.9 | |
| VideoLLaMA2-7BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 5.9 | 0 | 10 | 11.9 | 3.6 | 4.5 | 0 | 5.84 | |
| VideoLLaMA3-7BModel Category=Open-Source, Precision=16-bit, Evaluation Protocol=zero-shot, Frame Sampling=16 frames2026.05 | 0 | 5.9 | 0 | 10 | 14.3 | 3.6 | 9.1 | 10 | 7.79 |