Multi-discipline reasoning on MMMU (val) (Accuracy)
81.8AccuracyGPT-5
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-5Access=API call only2026.03 | 81.8 | |
| Gemini 2.5 ProAccess=API call only2026.03 | 79.6 | |
| Gemini 2.5 FlashAccess=API call only2026.03 | 79 | |
| GPT-5 miniAccess=API call only2026.03 | 78.7 | |
| Claude Sonnet 4.5Access=API call only2026.03 | 77.8 | |
| InternVL3.5-8BAccess=Open weights only2026.03 | 73.4 | |
| Keye-VL-1.5-8BAccess=Open weights only2026.03 | 71.4 | |
| Qwen3-VL-8BAccess=Open weights only2026.03 | 69.6 | |
| GLM-4.1V-9BAccess=Open weights only2026.03 | 68 | |
| MiniCPM-V-4.5-8BAccess=Open weights only2026.03 | 67.7 | |
| Qwen3-VL-32BBackbone=Qwen3-VL, Model Size=32B, Training Method=Baseline2026.05 | 67.7 | |
| iVGR-Qwen3-VL-32BBackbone=Qwen3-VL, Model Size=32B, Training Method=iVGR2026.05 | 67.7 | |
| Qwen3-VL-4BAccess=Open weights only2026.03 | 67.4 | |
| InternVL3.5-4BAccess=Open weights only2026.03 | 66.6 | |
| iVGR-Qwen3-VL-8BBackbone=Qwen3-VL, Model Size=8B, Training Method=iVGR2026.05 | 59.8 | |
| Qwen3-VL-8BBackbone=Qwen3-VL, Model Size=8B, Training Method=Baseline2026.05 | 58 | |
| Qwen3-VL-8BMode=base, Extractor=Regex letter extractor2026.05 | 56.8 | |
| S^3-FTBackbone=Qwen3-VL-8B, Mode=Single-view, Extractor=Regex letter extractor2026.05 | 56 | |
| Eagle2.5-8BAccess=Open weights only2026.03 | 55.8 | |
| iVGR-Qwen2.5-VL-7BBackbone=Qwen2.5-VL, Model Size=7B, Training Method=iVGR2026.05 | 55.2 | |
| S^3-FTBackbone=Qwen3-VL-8B, Mode=Cross-viewpoint, Extractor=Regex letter extractor2026.05 | 54.9 | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL, Model Size=7B, Training Method=Baseline2026.05 | 54.4 | |
| MolmoPoint-8BAccess=MolmoPoint2026.03 | 53.7 | |
| Molmo2-8BAccess=Fully open2026.03 | 53 | |
| Qwen2.5-VL-7BMode=base, Extractor=Regex letter extractor2026.05 | 52.4 | |
| S^3-FTBackbone=Qwen2.5-VL-7B, Mode=Single-view, Extractor=Regex letter extractor2026.05 | 51 | |
| Molmo2-4BAccess=Fully open2026.03 | 50.9 | |
| S^3-FTBackbone=Qwen2.5-VL-7B, Mode=Cross-viewpoint, Extractor=Regex letter extractor2026.05 | 49.6 | |
| PLM-8BAccess=Fully open2026.03 | 46.1 | |
| MolmoPoint-8B-O-7BAccess=Fully open2026.03 | 45.8 | |
| PLM-3BAccess=Fully open2026.03 | 41.2 | |
| Qwen2.5-VL-72B InstructZero-shot=true2026.02 | 37.2 | |
| Qwen2.5-VL-32B + AT-RL (Ours)Zero-shot=true2026.02 | 36.5 | |
| Qwen2.5-VL-32B + VPPOZero-shot=true2026.02 | 34.1 | |
| Gemini 2.0 FlashZero-shot=true2026.02 | 32.9 | |
| Qwen2.5-VL-32B InstructZero-shot=true2026.02 | 31.7 | |
| OpenAI GPT-4oZero-shot=true2026.02 | 31.1 | |
| Claude 3.5 SonnetZero-shot=true2026.02 | 30.5 |