Multi-image reasoning on Mantis (test)
72.81AccuracyDelimScaling
Evaluation Results
| Method | Links | |
|---|---|---|
| DelimScalingBackbone=InternVL3, Model Size=14B2026.02 | 72.81 | |
| InternVL3Model Size=14B, Method Variant=Baseline2026.02 | 71.89 | |
| DelimScalingBackbone=Qwen2.5-VL, Model Size=32B2026.02 | 70.05 | |
| DelimScalingBackbone=Qwen2.5-VL, Model Size=7B2026.02 | 69.12 | |
| DelimScalingBackbone=InternVL3, Model Size=8B2026.02 | 69.12 | |
| Qwen2.5-VLModel Size=7B, Method Variant=Baseline2026.02 | 68.66 | |
| Qwen2.5-VLModel Size=32B, Method Variant=Baseline2026.02 | 68.2 | |
| InternVL3Model Size=8B, Method Variant=Baseline2026.02 | 67.28 | |
| LLaVA-NeXT-Interleave-14BModel Scale=>14B2024.09 | 66.4 | |
| MM1.5-30BModel Scale=>14B2024.09 | 64.6 | |
| DelimScalingBackbone=LLaVA-OV, Model Size=7B2026.02 | 64.06 | |
| DelimScalingBackbone=Qwen2.5-VL, Model Size=3B2026.02 | 63.13 | |
| LLaVA-NeXT-Interleave-7BModel Scale=7B2024.09 | 62.7 | |
| GPT-4VModel Scale=>14B2024.09 | 62.7 | |
| LLaVA-OVModel Size=7B, Method Variant=Baseline2026.02 | 62.21 | |
| Qwen2.5-VLModel Size=3B, Method Variant=Baseline2026.02 | 59.91 | |
| MM1.5-7BModel Scale=7B2024.09 | 57.6 | |
| Mantis-Idefics2-8BModel Scale=7B2024.09 | 57.1 | |
| BLIP-3-4BModel Scale=3B2024.09 | 56.7 | |
| MM1.5-3BModel Scale=3B2024.09 | 54.8 | |
| MM1.5-3B-MoEModel Scale=3B2024.09 | 54.4 | |
| DelimScalingBackbone=InternVL3, Model Size=2B2026.02 | 54.38 | |
| MM1-30BModel Scale=>14B2024.09 | 52.5 | |
| InternVL3Model Size=2B, Method Variant=Baseline2026.02 | 52.07 | |
| MM1-7BModel Scale=7B2024.09 | 51.6 | |
| MM1.5-1B-MoEModel Scale=1B2024.09 | 51.2 | |
| MM1.5-1BModel Scale=1B2024.09 | 50.7 | |
| DelimScalingBackbone=InternVL3, Model Size=1B2026.02 | 49.77 | |
| Idefics2-8BModel Scale=7B2024.09 | 48.9 | |
| Phi-3-Vision-4BModel Scale=3B2024.09 | 47.9 | |
| InternVL3Model Size=1B, Method Variant=Baseline2026.02 | 47 | |
| LLaVA-NeXT-Interleave-0.5BModel Scale=1B2024.09 | 45.6 | |
| MM1-3BModel Scale=3B2024.09 | 45.2 | |
| MM1-1BModel Scale=1B2024.09 | 41.5 | |
| DelimScalingBackbone=LLaVA-OV, Model Size=0.5B2026.02 | 41.01 | |
| LLaVA-OVModel Size=0.5B, Method Variant=Baseline2026.02 | 40.09 | |
| LLaVAOneVision-0.5BModel Scale=1B2024.09 | 39.6 | |
| Emu2-Chat-37BModel Scale=>14B2024.09 | 37.8 | |
| LLaVA-v1.5-7BModel Scale=7B2024.09 | 31.3 |