Massive Multi-discipline Multimodal Understanding on MMMU
71.5AccuracySPOT-E
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| SPOT-EBase Model=Gemini-2.5-Flash, Source Type=Closed-Source, Test-time Adaptation=SPOT-E2026.06 | 71.5 | — | — | — | — | — | — | |
| SPOT-EBase Model=GPT-4o, Source Type=Closed-Source, Test-time Adaptation=SPOT-E2026.06 | 70.4 | — | — | — | — | — | — | |
| Gemini-2.5-FlashBase Model=Gemini-2.5-Flash, Source Type=Closed-Source, Test-time Adaptation=None2026.06 | 70 | — | — | — | — | — | — | |
| GPT-4oBase Model=GPT-4o, Source Type=Closed-Source, Test-time Adaptation=None2026.06 | 69.2 | — | — | — | — | — | — | |
| DARTAgent=Q, M, O2025.12 | 65.5 | — | — | — | — | — | — | |
| SPOT-EBase Model=InternVL3-8B, Source Type=Open-Source, Test-time Adaptation=SPOT-E2026.06 | 65.2 | — | — | — | — | — | — | |
| Debate with ConsensusAgent=Q, M, O2025.12 | 63.1 | — | — | — | — | — | — | |
| Debate with JudgeAgent=Q, M, O2025.12 | 63.1 | — | — | — | — | — | — | |
| InternVL3-8BBase Model=InternVL3-8B, Source Type=Open-Source, Test-time Adaptation=None2026.06 | 62.7 | — | — | — | — | — | — | |
| Qwen3-VL-8B-InstructBackbone=Qwen3-VL-8B, Strategy=Instruct2026.02 | 62.4 | — | — | — | — | — | — | |
| Qwen3-VL-8B-ThinkingBackbone=Qwen3-VL-8B, Strategy=LongCoT (Thinking)2026.02 | 62.3 | — | — | — | — | — | — | |
| SAPBackbone=Qwen3-VL-8B, Strategy=SAP2026.02 | 62.3 | — | — | — | — | — | — | |
| SPOT-EBase Model=GPT-4o-mini, Source Type=Closed-Source, Test-time Adaptation=SPOT-E2026.06 | 62 | — | — | — | — | — | — | |
| DARTAgent=Best Model2025.12 | 61.6 | — | — | — | — | — | — | |
| Qwen 3.5Zero-shot=true, Parameters=122B-A10B, Organization=Alibaba Cloud2026.05 | 61.22 | — | — | — | — | — | — | |
| SPOT-EBase Model=Qwen3-VL-8B, Source Type=Open-Source, Test-time Adaptation=SPOT-E2026.06 | 61 | — | — | — | — | — | — | |
| Self-Consistency (5-way)Agent=QwenVL2025.12 | 60.8 | — | — | — | — | — | — | |
| Phoenix-VL 1.5 MediumZero-shot=true, Parameters=123B, Organization=HTX2026.05 | 60.78 | — | — | — | — | — | — | |
| Debate with ConsensusAgent=3×QwenVL2025.12 | 60.5 | — | — | — | — | — | — | |
| Debate with JudgeAgent=3×QwenVL2025.12 | 60.3 | — | — | — | — | — | — | |
| GPT-4o-miniBase Model=GPT-4o-mini, Source Type=Closed-Source, Test-time Adaptation=None2026.06 | 60 | — | — | — | — | — | — | |
| SPOT-EBase Model=InternVL2.5-8B, Source Type=Open-Source, Test-time Adaptation=SPOT-E2026.06 | 59.5 | — | — | — | — | — | — | |
| Claude-3-OpusTable Header=Claude-O2026.02 | 59.4 | — | — | — | — | — | — | |
| CoTAgent=QwenVL2025.12 | 58.8 | — | — | — | — | — | — | |
| Gemini-1.52026.02 | 58.5 | — | — | — | — | — | — | |
| SPOT-EBase Model=Qwen2.5-VL-7B, Source Type=Open-Source, Test-time Adaptation=SPOT-E2026.06 | 58.5 | — | — | — | — | — | — | |
| Debate with JudgeAgent=3×Ovis22025.12 | 58.3 | — | — | — | — | — | — | |
| Qwen3-VL-8BBase Model=Qwen3-VL-8B, Source Type=Open-Source, Test-time Adaptation=None2026.06 | 58 | — | — | — | — | — | — | |
| Debate with ConsensusAgent=3×Ovis22025.12 | 57 | — | — | — | — | — | — | |
| GPT-4V2026.02 | 56.8 | — | — | — | — | — | — | |
| Self-RefinementAgent=QwenVL2025.12 | 56.5 | — | — | — | — | — | — | |
| InternVL2.5-8BBase Model=InternVL2.5-8B, Source Type=Open-Source, Test-time Adaptation=None2026.06 | 56 | — | — | — | — | — | — | |
| Qwen2.5-VL-7BBase Model=Qwen2.5-VL-7B, Source Type=Open-Source, Test-time Adaptation=None2026.06 | 55 | — | — | — | — | — | — | |
| Debate with JudgeAgent=3×MiniCPM-o2025.12 | 54.8 | — | — | — | — | — | — | |
| Qwen2-VLLLM Param=7B, Type=Und. Only2024.11 | 54.1 | — | — | — | — | — | — | |
| ViperGPTAgent=Qwen2025.12 | 54 | — | — | — | — | — | — | |
| Grok-1.5V2026.02 | 53.6 | — | — | — | — | — | — | |
| Debate with ConsensusAgent=3×MiniCPM-o2025.12 | 53.2 | — | — | — | — | — | — | |
| Full CacheBackbone=Qwen2.5-VL-7B-Instruct2025.11 | 53.18 | — | — | — | — | — | — | |
| H2Oretention ratio ρ=0.1, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 53.18 | — | — | — | — | — | — | |
| SnapKVretention ratio ρ=0.1, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 53.18 | — | — | — | — | — | — | |
| LOOK-Mretention ratio ρ=0.1, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 53.18 | — | — | — | — | — | — | |
| MEDAretention ratio ρ=0.1, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 53.18 | — | — | — | — | — | — | |
| FlashCacheretention ratio ρ=0.1, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 53.18 | — | — | — | — | — | — | |
| Claude-3-SonnetTable Header=Claude-S2026.02 | 53.1 | — | — | — | — | — | — | |
| Self-Consistency (5-way)Agent=Ovis22025.12 | 52.8 | — | — | — | — | — | — | |
| Self-Consistency (5-way)Agent=MiniCPM-o2025.12 | 52.5 | — | — | — | — | — | — | |
| FlashCacheretention ratio ρ=0.05, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 52.27 | — | — | — | — | — | — | |
| SnapKVretention ratio ρ=0.05, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 52.18 | — | — | — | — | — | — | |
| LOOK-Mretention ratio ρ=0.05, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 52.18 | — | — | — | — | — | — | |
| ChameleonAgent=Qwen2025.12 | 51.6 | — | — | — | — | — | — | |
| MEDAretention ratio ρ=0.05, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 51.36 | — | — | — | — | — | — | |
| H2Oretention ratio ρ=0.05, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 51.18 | — | — | — | — | — | — | |
| Qwen2.5-VL 7B, DenseBackbone=Qwen2.5-VL 7B, Sparsity=Dense2026.03 | 50.78 | — | — | — | — | — | — | |
| CoTAgent=Ovis22025.12 | 50.7 | — | — | — | — | — | — | |
| CoTAgent=MiniCPM-o2025.12 | 50.4 | — | — | — | — | — | — | |
| StreamingLLMretention ratio ρ=0.1, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 50 | — | — | — | — | — | — | |
| StreamingLLMretention ratio ρ=0.05, Backbone=Qwen2.5-VL-7B-Instruct2025.11 | 50 | — | — | — | — | — | — | |
| Gemma3-27BModel Size=27B2025.12 | 49.7 | — | — | — | — | — | — | |
| LLaVA-OneVision 7B, DenseBackbone=LLaVA-OneVision 7B, Sparsity=Dense2026.03 | 49.22 | — | — | — | — | — | — | |
| GLM-4.5VZero-shot=true, Parameters=106B-A12B, Organization=Z.ai2026.05 | 48.5 | — | — | — | — | — | — | |
| Self-RefinementAgent=Ovis22025.12 | 47.2 | — | — | — | — | — | — | |
| Self-RefinementAgent=MiniCPM-o2025.12 | 46.7 | — | — | — | — | — | — | |
| Phi-3-Vision-4BModel Scale=4B, Model Access Type=Open-source2025.03 | 46.1 | — | — | — | — | — | — | |
| Gemma3-4B + AuditDMModel Size=4B, Auditing Component=AuditDM2025.12 | 45.2 | — | — | — | — | — | — | |
| LWMLLM Param=7B, Type=Unified2024.11 | 44.8 | — | — | — | — | — | — | |
| Gemma3-12BModel Size=12B2025.12 | 44.8 | — | — | — | — | — | — | |
| Full CacheArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 43.34 | — | — | — | — | — | — | |
| PSFTTraining Stage=PSFT, RL Alignment (GRPO)=false2025.08 | 43.33 | — | — | — | — | — | — | |
| Llama 4 MaverickZero-shot=true, Parameters=400B-A17B, Organization=MetaAI2026.05 | 43 | — | — | — | — | — | — | |
| SMoE (k=2)k=2, Smoothing mechanism=None, Backbone=ViT-based, Model parameters=5.6B, Fine-tuning dataset=50% of the LLaVA-665K dataset, Number of experts=42026.06 | 42.67 | — | — | — | — | — | — | |
| SPOT-EBase Model=LLaVA-OV-7B, Source Type=Open-Source, Test-time Adaptation=SPOT-E2026.06 | 42.5 | — | — | — | — | — | — | |
| RetentiveKVArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 42.45 | — | — | — | — | — | — | |
| SmoothSMoE annealed (k=2)k=2, Smoothing mechanism=Annealed SmoothSMoE, Backbone=ViT-based, Model parameters=5.6B, Fine-tuning dataset=50% of the LLaVA-665K dataset, Number of experts=42026.06 | 42.44 | — | — | — | — | — | — | |
| H2OArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 42.39 | — | — | — | — | — | — | |
| MedaArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 42.36 | — | — | — | — | — | — | |
| SnapKVArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 42.33 | — | — | — | — | — | — | |
| Full CacheArchitecture=Qwen3-VL, Parameter Scale=8B2026.04 | 42.33 | — | — | — | — | — | — | |
| SAINTArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 42.28 | — | — | — | — | — | — | |
| Qwen2-VL-2BModel Scale=2B, Model Access Type=Open-source2025.03 | 42.2 | — | — | — | — | — | — | |
| LOOK-MArchitecture=LLaVA-v1.5, Parameter Scale=7B2026.04 | 42.15 | — | — | — | — | — | — | |
| Full CacheArchitecture=Qwen3-VL, Parameter Scale=4B2026.04 | 42.11 | — | — | — | — | — | — | |
| SmoothSMoE (k=2.5)k=2.5, Smoothing mechanism=SmoothSMoE, Backbone=ViT-based, Model parameters=5.6B, Fine-tuning dataset=50% of the LLaVA-665K dataset, Number of experts=42026.06 | 42.11 | — | — | — | — | — | — | |
| SmolVLM2-2.2BVision Encoder=ViT-SO400M, LLM=SmolLM2, #Visual Tokens=2106*, Vis. Enc. Size (M)=430, Evaluation Library=lmms-eval2024.12 | 42 | — | — | — | — | — | — | |
| BaseTraining Stage=Base, RL Alignment (GRPO)=false2025.08 | 41.89 | — | — | — | — | — | — | |
| FlorenceVL-3BVision Encoder=DaViT, LLM=Phi-3, #Visual Tokens=576, Vis. Enc. Size (M)=770, Evaluation Library=lmms-eval2024.12 | 41.8 | — | — | — | — | — | — | |
| MetaMorph#Params=8B, TFLOPs=1.12025.06 | 41.8 | — | — | — | — | — | — | |
| RetentiveKVArchitecture=Qwen3-VL, Parameter Scale=4B2026.04 | 41.66 | — | — | — | — | — | — | |
| RetentiveKVArchitecture=Qwen3-VL, Parameter Scale=8B2026.04 | 41.57 | — | — | — | — | — | — | |
| SnapKVArchitecture=Qwen3-VL, Parameter Scale=8B2026.04 | 41.38 | — | — | — | — | — | — | |
| MedaArchitecture=Qwen3-VL, Parameter Scale=4B2026.04 | 41.38 | — | — | — | — | — | — | |
| LOOK-MArchitecture=Qwen3-VL, Parameter Scale=4B2026.04 | 41.32 | — | — | — | — | — | — | |
| Self-Consistency (5-way)Agent=LLaVA 1.62025.12 | 41.3 | — | — | — | — | — | — | |
| SAINTArchitecture=Qwen3-VL, Parameter Scale=8B2026.04 | 41.22 | — | — | — | — | — | — | |
| SnapKVArchitecture=Qwen3-VL, Parameter Scale=4B2026.04 | 41.16 | — | — | — | — | — | — | |
| SAINTArchitecture=Qwen3-VL, Parameter Scale=4B2026.04 | 41.16 | — | — | — | — | — | — | |
| LOOK-MArchitecture=Qwen3-VL, Parameter Scale=8B2026.04 | 41.12 | — | — | — | — | — | — | |
| MedaArchitecture=Qwen3-VL, Parameter Scale=8B2026.04 | 41.1 | — | — | — | — | — | — | |
| Qwen2-VL-2B#Params=2B, TFLOPs=0.42025.06 | 41.1 | — | — | — | — | — | — | |
| LaTtE-Flow#Params=2B, TFLOPs=0.42025.06 | 41.1 | — | — | — | — | — | — |