Multi-image reasoning on Muirbench (test)
68AccuracyGPT-4V
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4VModel Scale=>14B, Note=Result is from GPT-4o as per table dagger symbol2024.09 | 68 | |
| Qwen2.5VLSize=7B, Type=AR, Samples=>9M2025.12 | 59.6 | |
| MM1.5-30BModel Scale=>14B2024.09 | 58.2 | |
| MM1.5-7BModel Scale=7B2024.09 | 49.1 | |
| LLaDA-VSize=8B, Type=Diff., Samples=16.5M2025.12 | 48.3 | |
| Qwen2.5VLSize=3B, Type=AR, Samples=>9M2025.12 | 47.7 | |
| DiffusionVLSize=3B, Type=Diff., Samples=738K2025.12 | 47.2 | |
| MM1.5-3B-MoEModel Scale=3B2024.09 | 45.6 | |
| DiffusionVLSize=7B, Type=Diff., Samples=738K2025.12 | 44.8 | |
| Mantis-Idefics2-8BModel Scale=7B2024.09 | 44.5 | |
| MM1.5-3BModel Scale=3B2024.09 | 44.3 | |
| LLaVA-OVSize=7B, Type=AR, Samples=7.8M2025.12 | 41.8 | |
| MM1.5-1B-MoEModel Scale=1B2024.09 | 40.9 | |
| LLaVA-NeXT-Interleave-7BModel Scale=7B2024.09 | 38.9 | |
| Phi-3-Vision-4BModel Scale=3B2024.09 | 38 | |
| MM1-30BModel Scale=>14B2024.09 | 36.7 | |
| MM1.5-1BModel Scale=1B2024.09 | 34.7 | |
| Emu2-Chat-37BModel Scale=>14B2024.09 | 33.6 | |
| MM1-1BModel Scale=1B2024.09 | 30.7 | |
| MM1-7BModel Scale=7B2024.09 | 30.4 | |
| MM1-3BModel Scale=3B2024.09 | 28 | |
| Idefics2-8BModel Scale=7B2024.09 | 26.1 | |
| LLaVAOneVision-0.5BModel Scale=1B2024.09 | 25.5 | |
| LLaVA-v1.5-7BModel Scale=7B2024.09 | 23.5 |