Multi-image difference analysis on MMXU (test)
23.4Worsen RateIDEFICS2
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| IDEFICS2Size=8B, Source=Open2025.02 | 23.4 | 57 | 66.8 | 49.1 | |
| DeepSeek-VLSize=1.3B, Source=Open2025.02 | 29.2 | 29.4 | 28.8 | 29.1 | |
| Qwen2-VLSize=7B, Source=Open2025.02 | 33.1 | 55.4 | 50 | 45.8 | |
| DeepSeek-VLSize=7B, Source=Open2025.02 | 38.9 | 31 | 60.6 | 43.5 | |
| InternVL2Size=8B, Source=Open2025.02 | 42.3 | 48.3 | 49.5 | 46.7 | |
| InternVL2Size=4B, Source=Open2025.02 | 47.6 | 57.1 | 31.5 | 45.4 | |
| GPT-4oSource=Closed2025.02 | 48 | 67.5 | 55.9 | 57.1 | |
| Claude-3.5Source=Closed2025.02 | 49.4 | 51.8 | 49.3 | 50.2 | |
| InternVL2Size=1B, Source=Open2025.02 | 55.3 | 21.4 | 11.3 | 29.3 | |
| InternVL2Size=2B, Source=Open2025.02 | 56 | 18.2 | 55.9 | 43.4 | |
| Llama3.2-visionSize=11B, Source=Open2025.02 | 59.6 | 60.8 | 35.6 | 52 | |
| Qwen2-VLSize=2B, Source=Open2025.02 | 71.2 | 25.8 | 28.4 | 41.8 |