Long-form multimodal understanding and reasoning on LongShOTBench 1.0 (test)
43.62Entity RecognitionGemini-2.5-Flash
Evaluation results
| Method | Links | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-2.5-FlashMode=Zero-shot2025.12 | 43.62 | 41.84 | 41.23 | 37.46 | 41.04 | 68.41 | 49.56 | 57.37 | 71.24 | 61.65 | 61.02 | 58.86 | 46.53 | 52.18 | 54.65 | 55.38 | 50.45 | 50.89 | 61.22 | 54.49 | 52.95 | 40.27 | |
| LongShOTAgentMode=Agentic Pipeline2025.12 | 42.84 | 35.41 | 31.35 | 35.51 | 36.28 | 54.26 | 45.16 | 45.52 | 52.87 | 49.45 | 48.87 | 60.17 | 38.47 | 33.7 | 45.3 | 44.15 | 37.79 | 44.26 | 64.15 | 47.59 | 44.66 | 38.25 | |
| Qwen3-VL-8B-InstructMode=Zero-shot2025.12 | 27.3 | 22.19 | 23.08 | 26.22 | 24.7 | 32.58 | 20.42 | 33.13 | 30.53 | 29.16 | 29.58 | 28.84 | 23.5 | 22.59 | 26.13 | 27.95 | 16.82 | 29.58 | 71.7 | 36.51 | 29.12 | — | |
| Qwen2.5-VL-7B-InstructMode=Zero-shot2025.12 | 20.54 | 14.21 | 14.11 | 9.07 | 14.48 | 24.76 | 13.34 | 19.56 | 20.72 | 19.59 | 18.78 | 18.61 | 14.62 | 13.1 | 16.28 | 19.14 | 10.81 | 21.05 | 40.57 | 22.89 | 18.39 | — | |
| InternVL3.5-8BMode=Zero-shot2025.12 | 19.95 | 14.47 | 15.88 | 12.36 | 15.66 | 23.98 | 14.64 | 22.14 | 20.05 | 20.2 | 21.41 | 22.72 | 15.91 | 14.03 | 18.52 | 18.99 | 12.31 | 24.23 | 54.72 | 27.56 | 20.49 | — | |
| Qwen2.5-Omni-7BMode=Zero-shot2025.12 | 17.03 | 13.95 | 14.17 | 16.2 | 15.34 | 23.73 | 12.92 | 19.53 | 17.93 | 18.53 | 18.6 | 19.16 | 13.04 | 15.23 | 16.51 | 16.59 | 11.21 | 22.41 | 42.45 | 23.17 | 18.39 | — | |
| LLaVA-NeXT-Video-7B-hfMode=Zero-shot2025.12 | 11.16 | 9.6 | 10.34 | 9.53 | 10.16 | 14.43 | 2.92 | 14.6 | 13.61 | 11.39 | 13.42 | 18.22 | 9.9 | 7.86 | 12.35 | 11.34 | 10.91 | 15.19 | 47.17 | 13.82 | 13.76 | — | |
| LLaVA-OneVision-Qwen2-7B-ovMode=Zero-shot2025.12 | 8.12 | 6.66 | 7.5 | 6.08 | 7.09 | 9.01 | 1.79 | 11.7 | 9.73 | 8.06 | 9.14 | 12.92 | 8.33 | 5.31 | 8.92 | 8.92 | 4.6 | 9.33 | 16.98 | 9.96 | 8.51 | — |