Video Understanding on VideoMMMU
68.64AccuracyVanilla MoE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Vanilla MoEBackbone=QWEN3-VL-30B-A3B-INSTRUCT2026.04 | 68.64 | — | — | — | — | — | |
| MACS (Ours)Backbone=QWEN3-VL-30B-A3B-INSTRUCT2026.04 | 68.49 | — | — | — | — | — | |
| MACS (w/o Expanded)Backbone=QWEN3-VL-30B-A3B-INSTRUCT2026.04 | 67.92 | — | — | — | — | — | |
| VideoSearcher-8BEvaluation Setting=Agentic Model2026.07 | 66.33 | — | — | — | — | — | |
| Qwen3-VL-8B-InstructEvaluation Setting=Direct Answer2026.07 | 65.67 | — | — | — | — | — | |
| VideoSearcher-4BEvaluation Setting=Agentic Model2026.07 | 65.33 | — | — | — | — | — | |
| Vanilla MoEBackbone=INTERNVL3.5-30B-A3B2026.04 | 65.24 | — | — | — | — | — | |
| MACS (Ours)Backbone=INTERNVL3.5-30B-A3B2026.04 | 65.03 | — | — | — | — | — | |
| MACS (w/o Expanded)Backbone=INTERNVL3.5-30B-A3B2026.04 | 64.58 | — | — | — | — | — | |
| CAI-MoE (Expanded)Backbone=QWEN3-VL-30B-A3B-INSTRUCT2026.04 | 64.28 | — | — | — | — | — | |
| CAI-MoE (Token Drop)Backbone=QWEN3-VL-30B-A3B-INSTRUCT2026.04 | 62.17 | — | — | — | — | — | |
| GPT-4o2025.12 | 61.2 | — | — | — | — | — | |
| GPT-4o2026.05 | 61.2 | — | — | — | — | — | |
| CAI-MoE (Expanded)Backbone=INTERNVL3.5-30B-A3B2026.04 | 61.17 | — | — | — | — | — | |
| Qwen3-VL-4B-InstructEvaluation Setting=Direct Answer2026.07 | 60.78 | — | — | — | — | — | |
| CAI-MoE (Token Drop)Backbone=INTERNVL3.5-30B-A3B2026.04 | 58.63 | — | — | — | — | — | |
| Vanilla MoEBackbone=KIMI-VL-A3B-INSTRUCT2026.04 | 57.58 | — | — | — | — | — | |
| MACS (Ours)Backbone=KIMI-VL-A3B-INSTRUCT2026.04 | 57.49 | — | — | — | — | — | |
| MACS (w/o Expanded)Backbone=KIMI-VL-A3B-INSTRUCT2026.04 | 57.12 | — | — | — | — | — | |
| AdaTooler-V-7BFrames=1282025.12 | 56.8 | — | — | — | — | — | |
| AdaTooler-V-7BFrames=642025.12 | 55.1 | — | — | — | — | — | |
| CAI-MoE (Expanded)Backbone=KIMI-VL-A3B-INSTRUCT2026.04 | 54.63 | — | — | — | — | — | |
| AdaTooler-V-7BFrames=322025.12 | 54.6 | — | — | — | — | — | |
| Gemini 1.5 Pro2025.12 | 53.9 | — | — | — | — | — | |
| CAI-MoE (Token Drop)Backbone=KIMI-VL-A3B-INSTRUCT2026.04 | 52.87 | — | — | — | — | — | |
| Video-RTS-7B2026.05 | 52.7 | — | — | — | — | — | |
| Video-R12025.12 | 52.4 | — | — | — | — | — | |
| Video-R1Evaluation Setting=Direct Answer2026.07 | 52.4 | — | — | — | — | — | |
| Video-XL2025.12 | 52.3 | — | — | — | — | — | |
| Open-o3 Video2025.12 | 52.3 | — | — | — | — | — | |
| ViSS-R1-7B2026.05 | 51.7 | — | — | — | — | — | |
| Qwen2.5-VL-7B + SFT + SAGESFT=true, SAGE=true2026.05 | 51.7 | — | — | — | — | — | |
| VideoChat-R1.5Evaluation Setting=Direct Answer2026.07 | 51.4 | — | — | — | — | — | |
| Qwen2.5-VL-7B-InstructFrames=1282025.12 | 51.3 | — | — | — | — | — | |
| VideoRFT-7B2026.05 | 51.1 | — | — | — | — | — | |
| VideoRFTEvaluation Setting=Direct Answer2026.07 | 51.1 | — | — | — | — | — | |
| Qwen2.5-VL-7B + SAGESAGE=true2026.05 | 50.6 | — | — | — | — | — | |
| VideoCoMEvaluation Setting=Direct Answer2026.07 | 50.2 | — | — | — | — | — | |
| Video-R1-7B2026.05 | 49.8 | — | — | — | — | — | |
| VidGroundFrames=642026.04 | 49.4 | — | — | — | — | — | |
| Qwen2.5-VL-7B-InstructFrames=642025.12 | 49.1 | — | — | — | — | — | |
| TW-GRPOFrames=162026.04 | 48.6 | — | — | — | — | — | |
| VidGroundFrames=322026.04 | 48.3 | — | — | — | — | — | |
| TW-GRPOFrames=642026.04 | 48.3 | — | — | — | — | — | |
| TW-GRPOFrames=322026.04 | 47.9 | — | — | — | — | — | |
| Qwen2.5-VL-7B-SFTFrames=322026.04 | 47.8 | — | — | — | — | — | |
| Video-RTSFrames=322026.04 | 47.7 | — | — | — | — | — | |
| Qwen2.5-VL-7B-SFTFrames=642026.04 | 47.6 | — | — | — | — | — | |
| Qwen2.5-VL-7B-InstructFrames=322025.12 | 47.4 | — | — | — | — | — | |
| VidGroundFrames=162026.04 | 47.4 | — | — | — | — | — | |
| Video-RTSFrames=162026.04 | 47.1 | — | — | — | — | — | |
| Qwen2.5-VL-7BFrames=642026.04 | 46.6 | — | — | — | — | — | |
| Video-RTSFrames=642026.04 | 46.4 | — | — | — | — | — | |
| Qwen2.5-VL-7B2026.05 | 46.1 | — | — | — | — | — | |
| Qwen2.5-VL-7BFrames=322026.04 | 45.4 | — | — | — | — | — | |
| Video-R1Frames=322026.04 | 45.4 | — | — | — | — | — | |
| Video-R1Frames=642026.04 | 45.4 | — | — | — | — | — | |
| Qwen2.5-VL-7BFrames=162026.04 | 45 | — | — | — | — | — | |
| Video-R1Frames=162026.04 | 44.7 | — | — | — | — | — | |
| Qwen2.5-VL-7B + SFTSFT=true2026.05 | 43.4 | — | — | — | — | — | |
| Qwen2.5-VL-7B-SFTFrames=162026.04 | 43.1 | — | — | — | — | — | |
| LongVILA-R1-7BFrames=642026.04 | 41.2 | — | — | — | — | — | |
| LongVILA-R1-7BFrames=322026.04 | 40.7 | — | — | — | — | — | |
| LongVILA-R1-7BFrames=162026.04 | 38.8 | — | — | — | — | — | |
| VILA-1.5-40B2026.05 | 34 | — | — | — | — | — | |
| LLaVA-OneVision-7B2026.05 | 33.8 | — | — | — | — | — | |
| LongVA-7B2026.05 | 23.9 | — | — | — | — | — | |
| DELTAVID-Qwen3-VL-8BParameters=8B, Base Model=Qwen3-VL2026.06 | — | — | — | — | 64.56 | — | |
| EgoR1Model Category=Tool-calling, RAG=false2026.03 | — | — | — | — | 34.4 | 37.6 | |
| Gemini 2.5 Pro2026.03 | — | — | — | — | 83.6 | — | |
| Gemini-1.5-Pro2026.02 | — | 49.3 | 53.3 | 59 | 53.8 | — | |
| Gemini-3-Pro2026.03 | — | — | — | — | 87.6 | — | |
| GPT-4o2026.02 | — | 55.6 | 62 | 66 | 61.2 | — | |
| GPT-4oModel Category=Proprietary2026.03 | — | — | — | — | 61.2 | 66 | |
| LongVT-7B-RFTParameters=7B2026.02 | — | 35.7 | 43.7 | 56.7 | 45.3 | — | |
| LongVT-RL-7BModel Category=Tool-calling, Model Scale=7B2026.03 | — | — | — | — | 42.6 | 50 | |
| Open-o3-video-7BModel Category=Tool-free, Model Scale=7B2026.03 | — | — | — | — | 52.3 | 68 | |
| Qwen2.5-VL-7BParameters=7B, reproduced=true2026.02 | — | 35.9 | 36.1 | 57.6 | 43.2 | — | |
| Qwen2.5-VL-7BModel Category=Tool-free, Model Scale=7B2026.03 | — | — | — | — | 51.2 | 64.7 | |
| Qwen3-VL-8BParameters=8B2026.06 | — | — | — | — | 57 | — | |
| Seed-1.5-VL2026.03 | — | — | — | — | 81.4 | — | |
| Seed1.82026.03 | — | — | — | — | 82.7 | — | |
| Video-R12026.06 | — | — | — | — | 52.4 | — | |
| Video-R1-7BParameters=7B, reproduced=true2026.02 | — | 44.2 | 46 | 67.3 | 52.4 | — | |
| VideoChat-R1-7BParameters=7B, reproduced=true2026.02 | — | 40.2 | 45.6 | 53 | 46.8 | — | |
| VideoR1-7BModel Category=Tool-free, Model Scale=7B2026.03 | — | — | — | — | 52.4 | 65.3 | |
| VideoRFT-7BParameters=7B2026.02 | — | — | — | — | 51.1 | — | |
| VideoRFT-7BModel Category=Tool-free, Model Scale=7B2026.03 | — | — | — | — | 51.1 | 66 | |
| VideoTemp-o3-7B-RLParameters=7B2026.02 | — | 43 | 47.8 | 69 | 53.2 | — | |
| VideoTemp-o3-7B-SFTParameters=7B2026.02 | — | 39.3 | 46.4 | 60.4 | 48.7 | — | |
| VILA-1.5-40BParameters=40B2026.02 | — | 32.6 | 30.6 | 38.6 | 34 | — | |
| VISIONCOACH-7BModel Category=Tool-free, Model Scale=7B2026.03 | — | — | — | — | 54.4 | 70.3 |