Vision-related Reasoning and Perception on MMBench (test)
80.38Avg ScoreRotation
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RotationCategory=SSL4RL2025.10 | 80.38 | — | 83.89 | 80.54 | — | — | — | 65.84 | 80.21 | 71.53 | 84.76 | |
| PositionCategory=SSL4RL2025.10 | 80.08 | — | 82.22 | 77.19 | — | — | — | 67.65 | 82.15 | 66.51 | 85.39 | |
| JigsawCategory=SSL4RL2025.10 | 77.82 | — | 80.35 | 74.51 | — | — | — | 62.86 | 77.92 | 67.82 | 84.31 | |
| Idefics2Size=8B, # tokens per image=64, Zero-shot=true2024.05 | 76.8 | — | — | — | — | — | — | — | — | — | — | |
| Idefics2Size=8B, # tokens per image=320, Zero-shot=true2024.05 | 76.7 | — | — | — | — | — | — | — | — | — | — | |
| InternLM-XComposer2023.09 | 74.4 | 50.6 | 82 | 76.1 | 79.3 | 59.2 | 81.7 | — | — | — | — | |
| DeepSeek-VLSize=7B, # tokens per image=576, Zero-shot=true2024.05 | 73.2 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL-3BCategory=Base2025.10 | 72.99 | — | 76.62 | 41.54 | — | — | — | 61.77 | 73.55 | 64.32 | 82.06 | |
| MM1-ChatSize=7B, # tokens per image=720, Zero-shot=true2024.05 | 72.3 | — | — | — | — | — | — | — | — | — | — | |
| LLaVA-NeXTSize=13B, # tokens per image=2880, Zero-shot=true2024.05 | 70 | — | — | — | — | — | — | — | — | — | — | |
| ContrastiveCategory=SSL4RL2025.10 | 69.27 | — | 71.81 | 73.42 | — | — | — | 61.12 | 65.38 | 58.39 | 78.5 | |
| mPLUG-Owl2023.09 | 66 | 43.4 | 76 | 62.1 | 68.6 | 55.9 | 73 | — | — | — | — | |
| MMICL2023.09 | 65.2 | 44.3 | 77.9 | 64.8 | 66.5 | 53.6 | 70.6 | — | — | — | — | |
| LMEye2023.09 | 62.6 | 41 | 74.3 | 55.9 | 61.6 | 58.7 | 69.2 | — | — | — | — | |
| Qwen-VL-Chat2023.09 | 61.8 | 40.5 | 74.3 | 47.9 | 66.3 | 46.2 | 72.8 | — | — | — | — | |
| Shikra2023.09 | 60.2 | 33.5 | 69.6 | 53.1 | 61.8 | 50.4 | 71.7 | — | — | — | — | |
| LLaVA-1.52023.09 | 59.5 | 32.4 | 72.6 | 49.3 | 62.3 | 52.2 | 67.7 | — | — | — | — | |
| Otter-I2023.09 | 48.3 | 22.2 | 63.3 | 39.4 | 46.8 | 36.4 | 60.6 | — | — | — | — | |
| LLaMA-A-V22023.09 | 38.9 | 7.4 | 45.3 | 19.2 | 45 | 32 | 54 | — | — | — | — | |
| LLaVA2023.09 | 36.2 | 15.9 | 53.6 | 28.6 | 41.8 | 20 | 40.4 | — | — | — | — | |
| InstructBLIP2023.09 | 33.9 | 21.6 | 47.4 | 22.5 | 33 | 24.4 | 41.1 | — | — | — | — | |
| VisualGLM2023.09 | 33.5 | 11.4 | 48.8 | 27.7 | 35.8 | 17.6 | 41.5 | — | — | — | — | |
| MiniGPT-42023.09 | 23 | 13.6 | 32.9 | 8.9 | 28.8 | 11.2 | 28.3 | — | — | — | — |