Multimodal Reasoning on MMMU (Efficiency and Performance Metrics)
57.1MMMU ScoreDPA-Qwen3-32B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DPA-Qwen3-32BTarget LLM=Qwen3-32B2026.05 | 57.1 | — | — | — | |
| LLaVA-NeXT-Qwen3-32BTarget LLM=Qwen3-32B2026.05 | 56.9 | — | — | — | |
| FastMMoE*Retention Ratio (Vision Tokens)=75%2025.11 | 51.56 | 72.49 | 0.19 | 21.09 | |
| FastMMoE†Retention Ratio (Vision Tokens)=50%, lv=2, Kv=22025.11 | 51.56 | 70.57 | 2.11 | 61.76 | |
| FastMMoE†Retention Ratio (Vision Tokens)=75%, lv=10, Kv=22025.11 | 51.44 | 72 | 0.68 | 39.15 | |
| BaselineRetention Ratio (Vision Tokens)=100%2025.11 | 51.33 | 72.68 | 0 | 0 | |
| SparseVLMRetention Ratio (Vision Tokens)=75%2025.11 | 51.11 | 71.72 | 0.96 | — | |
| FastMMoE*Retention Ratio (Vision Tokens)=50%2025.11 | 51.11 | 71.94 | 0.74 | 43.07 | |
| FastVRetention Ratio (Vision Tokens)=50%2025.11 | 51 | 71.91 | 0.77 | — | |
| FastVRetention Ratio (Vision Tokens)=75%2025.11 | 50.67 | 72.18 | 0.5 | — | |
| FastVRetention Ratio (Vision Tokens)=25%2025.11 | 50.56 | 70.02 | 2.65 | — | |
| FastMMoE*Retention Ratio (Vision Tokens)=25%2025.11 | 50.56 | 70.32 | 2.36 | 63.66 | |
| DPA-Qwen3-4BTarget LLM=Qwen3-4B2026.05 | 50 | — | — | — | |
| LLaVA-NeXT-Qwen3-4BTarget LLM=Qwen3-4B2026.05 | 49.9 | — | — | — | |
| SparseVLMRetention Ratio (Vision Tokens)=50%2025.11 | 49.33 | 70.31 | 2.37 | — | |
| SparseVLMRetention Ratio (Vision Tokens)=25%2025.11 | 48.11 | 66.61 | 6.07 | — | |
| Idefics2-8BTarget LLM=Public Models2026.05 | 43.1 | — | — | — | |
| Qwen2-VL-2BTarget LLM=Public Models2026.05 | 41.1 | — | — | — | |
| Cambrian-1-8BTarget LLM=Public Models2026.05 | 40.8 | — | — | — | |
| DPA-LLaMA-3.2-3BTarget LLM=LLaMA-3.2-3B2026.05 | 37.8 | — | — | — | |
| UnicornText-only=true, Pretrain Scale=1.2M2025.03 | 36.87 | — | — | — | |
| LLaVA-NeXT-7BTarget LLM=Public Models2026.05 | 35.1 | — | — | — | |
| LLaVA-1.5-7BTarget LLM=Public Models2026.05 | 34.9 | — | — | — | |
| BunnyText-only=false, Pretrain Scale=1.2M2025.03 | 34.62 | — | — | — | |
| Unicorn*Text-only=true, Pretrain Scale=558K2025.03 | 34.23 | — | — | — | |
| LLaVA-1.5Text-only=false, Pretrain Scale=558K2025.03 | 33.16 | — | — | — | |
| LLaVA-NeXT-LLaMA-3.2-3BTarget LLM=LLaMA-3.2-3B2026.05 | 32.9 | — | — | — |