Multi-image reasoning on QBench2 (val)
79.3AccuracyMM1.5-30B
Evaluation Results
| Method | Links | |
|---|---|---|
| MM1.5-30BModel Scale=>14B2024.09 | 79.3 | |
| LLaVA-NeXT-Interleave-14BModel Scale=>14B2024.09 | 76.7 | |
| GPT-4VModel Scale=>14B2024.09 | 76.5 | |
| Mantis-Idefics2-8BModel Scale=7B2024.09 | 75.2 | |
| BLIP-3-4BModel Scale=3B2024.09 | 75.1 | |
| LLaVA-NeXT-Interleave-7BModel Scale=7B2024.09 | 74.2 | |
| MM1.5-3B-MoEModel Scale=3B2024.09 | 73.8 | |
| MM1.5-3BModel Scale=3B2024.09 | 73.2 | |
| MM1.5-7BModel Scale=7B2024.09 | 73.2 | |
| MM1.5-1B-MoEModel Scale=1B2024.09 | 70.9 | |
| MM1.5-1BModel Scale=1B2024.09 | 66.4 | |
| Idefics2-8BModel Scale=7B2024.09 | 57 | |
| Phi-3-Vision-4BModel Scale=3B2024.09 | 56.8 | |
| LLaVA-NeXT-Interleave-0.5BModel Scale=1B2024.09 | 52 | |
| Emu2-Chat-37BModel Scale=>14B2024.09 | 50.1 | |
| LLaVA-v1.5-7BModel Scale=7B2024.09 | 49.3 | |
| LLaVAOneVision-0.5BModel Scale=1B2024.09 | 48.8 | |
| MM1-7BModel Scale=7B2024.09 | 43.6 | |
| MM1-1BModel Scale=1B2024.09 | 43.4 | |
| MM1-30BModel Scale=>14B2024.09 | 42.8 | |
| MM1-3BModel Scale=3B2024.09 | 41.4 |