Video Question Answering on VideoMME (test)
89.8Short Length AccuracyLSCI
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LSCIparameter_count=7B, coverage=solve 33%2024.12 | 89.8 | 78.6 | 77.7 | 74.8 | — | |
| GPT-4oinput=subtitle2024.12 | 82.8 | 77.1 | 76.6 | 72.1 | — | |
| Qwen3-VL + Q-GateBackbone=Qwen3-VL, Strategy=Q-Gate, Frame=322026.04 | 79.41 | — | 66.13 | 61.19 | — | |
| FOCUS (LLaVA-V-7B)Frame=642026.04 | 76.5 | — | 63.5 | 56.1 | — | |
| Baselineinput=subtitle2024.12 | 76.2 | 64.4 | 63.4 | 53.4 | — | |
| Baseline + LSCI2024.12 | 75.7 | 66.2 | 64 | 58.9 | — | |
| Qwen3-VL + AKS*Backbone=Qwen3-VL, Strategy=AKS*, Frame=322026.04 | 75.68 | — | 62.9 | 54.79 | — | |
| Qwen3-VL + T*Backbone=Qwen3-VL, Strategy=T*, Frame=322026.04 | 75 | — | 58.57 | 54.22 | — | |
| Gemini-1.5-Pro (0615)Frame=1 fps2026.04 | 75 | — | 74.3 | 67.4 | — | |
| Qwen3-VL + VSLSBackbone=Qwen3-VL, Strategy=VSLS, Frame=322026.04 | 74.35 | — | 59.81 | 54.68 | — | |
| Baselineparameter_count=7B2024.12 | 73.8 | 62.8 | 62.3 | 52.3 | — | |
| Qwen3-VL + Q-GateBackbone=Qwen3-VL, Strategy=Q-Gate, Frame=82026.04 | 73.04 | — | 61.29 | 57.19 | — | |
| Uniform SamplingBackbone Model=GPT-4o, Initial Keyframe Setting=Top 32, Avg. Frames=32.02025.08 | 71.4 | — | 61 | 55.2 | — | |
| LLaVA-OneVision-7BSize=7B, Tokens (%)=1002026.03 | 70.1 | 58.5 | 56.6 | 48.8 | — | |
| VanillaSize=7B, Tokens (%)=1002026.03 | 70.1 | 58.5 | 56.6 | 48.8 | — | |
| FastVIDSize=7B, Tokens (%)=25.02026.03 | 69.9 | 58 | 56.6 | 47.7 | — | |
| Q-Frame (GPT-4o)Frame=82026.04 | 69.9 | — | 63.8 | 57.6 | — | |
| GPT-4o + Q-GateBackbone=GPT-4o, Strategy=Q-Gate, Frame=322026.04 | 69.44 | — | 62.77 | 60.05 | — | |
| FastVIDSize=7B, Tokens (%)=15.02026.03 | 69.3 | 57.7 | 56.2 | 47.4 | — | |
| Uniform SamplingBackbone Model=GPT-4o, Initial Keyframe Setting=Top 8, Avg. Frames=8.02025.08 | 69.2 | — | 59.3 | 54.3 | — | |
| VisionZipSize=7B, Tokens (%)=25.02026.03 | 68.9 | 58 | 57.4 | 47.6 | — | |
| PruneVidSize=7B, Tokens (%)=25.02026.03 | 68.8 | 57 | 54.4 | 47.7 | — | |
| Qwen3-VL + AKS*Backbone=Qwen3-VL, Strategy=AKS*, Frame=82026.04 | 68.79 | — | 58.33 | 54.45 | — | |
| SemVIDSize=7B, Tokens (%)=25.02026.03 | 68.7 | 58.1 | 57 | 48.7 | — | |
| GPT-4o (Uniform)Backbone=GPT-4o, Strategy=Uniform, Frame=322026.04 | 68.63 | — | 52.96 | 52.4 | — | |
| SemVIDSize=7B, Tokens (%)=15.02026.03 | 68.3 | 57.9 | 56.7 | 48.7 | — | |
| FrameFusionSize=7B, Tokens (%)=25.02026.03 | 68.2 | 57.5 | 55.7 | 48.6 | — | |
| PruneVidSize=7B, Tokens (%)=15.02026.03 | 67.9 | 56.8 | 54.3 | 48.1 | — | |
| GPT-4o + Q-GateBackbone=GPT-4o, Strategy=Q-Gate, Frame=82026.04 | 67.16 | — | 59.68 | 54.78 | — | |
| GPT-4o + AKS*Backbone=GPT-4o, Strategy=AKS*, Frame=82026.04 | 66.85 | — | 57.86 | 54 | — | |
| GPT-4o + AKS*Backbone=GPT-4o, Strategy=AKS*, Frame=322026.04 | 66.63 | — | 59.2 | 52.89 | — | |
| Qwen3-VL + VSLSBackbone=Qwen3-VL, Strategy=VSLS, Frame=82026.04 | 66.5 | — | 53.76 | 51.37 | — | |
| LLaVA-OneVision-72BFrame=322026.04 | 66.3 | — | 62.2 | 60 | — | |
| GPT-4o + T*Backbone=GPT-4o, Strategy=T*, Frame=322026.04 | 66.28 | — | 56.01 | 51.48 | — | |
| FastVSize=7B, Tokens (%)=25.02026.03 | 66 | 55.9 | 54.6 | 47.2 | — | |
| Qwen3-VL (Uniform)Backbone=Qwen3-VL, Strategy=Uniform, Frame=322026.04 | 65.85 | — | 54.57 | 51.6 | — | |
| FrameFusionSize=7B, Tokens (%)=15.02026.03 | 65.8 | 55.5 | 54.1 | 46.7 | — | |
| Qwen3-VL + T*Backbone=Qwen3-VL, Strategy=T*, Frame=82026.04 | 65.79 | — | 55.6 | 52.28 | — | |
| AFP+GraphBackbone Model=GPT-4o, Initial Keyframe Setting=Top 32, Avg. Frames=4.22025.08 | 65.4 | — | 56.7 | 55 | — | |
| AFP+GraphBackbone Model=GPT-4o, Initial Keyframe Setting=Top 8, Avg. Frames=2.22025.08 | 64.4 | — | 55.5 | 52.8 | — | |
| GPT-4o + VSLSBackbone=GPT-4o, Strategy=VSLS, Frame=322026.04 | 64.05 | — | 55.11 | 50.68 | — | |
| GPT-4o + VSLSBackbone=GPT-4o, Strategy=VSLS, Frame=82026.04 | 63.82 | — | 55.07 | 50.67 | — | |
| VisionZipSize=7B, Tokens (%)=15.02026.03 | 63.8 | 55.5 | 54.4 | 48.3 | — | |
| GPT-4o (Uniform)Backbone=GPT-4o, Strategy=Uniform, Frame=82026.04 | 63.4 | — | 52.15 | 50.12 | — | |
| Qwen3-VL (Uniform)Backbone=Qwen3-VL, Strategy=Uniform, Frame=82026.04 | 63.4 | — | 52.15 | 51.14 | — | |
| GPT-4o + T*Backbone=GPT-4o, Strategy=T*, Frame=82026.04 | 60.36 | — | 53.58 | 51.26 | — | |
| Uniform SamplingBackbone Model=Qwen2.5-VL-7B, Initial Keyframe Setting=Top 32, Avg. Frames=32.02025.08 | 60 | — | 38.1 | 33.6 | — | |
| FastVSize=7B, Tokens (%)=15.02026.03 | 58.4 | 51.9 | 51.7 | 45.4 | — | |
| AKS*Backbone Model=GPT-4o, Initial Keyframe Setting=Top 32, Avg. Frames=32.02025.08 | 57.8 | — | 54.5 | 55.6 | — | |
| AKS*Backbone Model=GPT-4o, Initial Keyframe Setting=Top 8, Avg. Frames=8.02025.08 | 55.8 | — | 52.7 | 51.6 | — | |
| AFP+GraphBackbone Model=LLaVA-Video-7B, Initial Keyframe Setting=Top 8, Avg. Frames=2.12025.08 | 55.2 | — | 46 | 43.9 | — | |
| AFP+GraphBackbone Model=LLaVA-Video-7B, Initial Keyframe Setting=Top 32, Avg. Frames=4.22025.08 | 53.9 | — | 46.6 | 42.6 | — | |
| AFP+GraphBackbone Model=Qwen2.5-VL-7B, Initial Keyframe Setting=Top 32, Avg. Frames=4.22025.08 | 50.9 | — | 41.3 | 39.2 | — | |
| AFP+GraphBackbone Model=Qwen2.5-VL-7B, Initial Keyframe Setting=Top 8, Avg. Frames=2.12025.08 | 48.9 | — | 37.1 | 37 | — | |
| AKS*Backbone Model=Qwen2.5-VL-7B, Initial Keyframe Setting=Top 32, Avg. Frames=32.02025.08 | 43.8 | — | 39.7 | 37.4 | — | |
| AKS*Backbone Model=Qwen2.5-VL-7B, Initial Keyframe Setting=Top 8, Avg. Frames=8.02025.08 | 42.2 | — | 37.8 | 39.4 | — | |
| AKS*Backbone Model=LLaVA-Video-7B, Initial Keyframe Setting=Top 32, Avg. Frames=32.02025.08 | 41.6 | — | 40.9 | 39.6 | — | |
| Uniform SamplingBackbone Model=LLaVA-Video-7B, Initial Keyframe Setting=Top 8, Avg. Frames=8.02025.08 | 41.4 | — | 41.1 | 37.4 | — | |
| Uniform SamplingBackbone Model=Qwen2.5-VL-7B, Initial Keyframe Setting=Top 8, Avg. Frames=8.02025.08 | 41.1 | — | 39.1 | 36.6 | — | |
| AKS*Backbone Model=LLaVA-Video-7B, Initial Keyframe Setting=Top 8, Avg. Frames=8.02025.08 | 40.5 | — | 42.2 | 38 | — | |
| Uniform SamplingBackbone Model=LLaVA-Video-7B, Initial Keyframe Setting=Top 32, Avg. Frames=32.02025.08 | 39.9 | — | 41.7 | 39.1 | — | |
| Cambrian-4BKV size=4K2026.05 | — | — | — | — | 61.4 | |
| Cambrian-4BKV size=20K2026.05 | — | — | — | — | 65.9 | |
| Cambrian-4B + Gaze attentionKV size=4K2026.05 | — | — | — | — | 67.5 | |
| Cambrian-4B + Gaze attentionKV size=2K2026.05 | — | — | — | — | 67.1 | |
| Cambrian-4B + HERMESKV size=4K2026.05 | — | — | — | — | 65 | |
| Cambrian-7BKV size=10K2026.05 | — | — | — | — | 66.5 | |
| Cambrian-7B + Gaze attentionKV size=1.5K2026.05 | — | — | — | — | 67 | |
| Cambrian-7B + HERMESKV size=1.5K2026.05 | — | — | — | — | 66.3 | |
| Gemini 2.0 FlashKV size=–2026.05 | — | — | — | — | 70.3 | |
| GPT-4oKV size=–2026.05 | — | — | — | — | 77.2 | |
| InternVL2.5-4BKV size=–2026.05 | — | — | — | — | 62.3 | |
| LLaVA-OV-7BKV size=25K2026.05 | — | — | — | — | 64.7 | |
| PLM-3BKV size=–2026.05 | — | — | — | — | 54.9 | |
| Qwen2-VL-7BKV size=50K2026.05 | — | — | — | — | 63.9 | |
| Qwen2-VL-7B + InfiniPot-VKV size=6K2026.05 | — | — | — | — | 65.8 | |
| Qwen2-VL-7B + StreamMemKV size=6K2026.05 | — | — | — | — | 65.9 | |
| Qwen2.5-VL-3BKV size=50K2026.05 | — | — | — | — | 63.3 | |
| Qwen2.5-VL-3B + InfiniPot-VKV size=6K2026.05 | — | — | — | — | 62.1 | |
| Qwen2.5-VL-3B + StreamMemKV size=6K2026.05 | — | — | — | — | 62.3 |