Multi-modal Question Answering on MMStar (test)
72.7AccuracyProprietary API SOTA (SenseTime, 2024)
Evaluation results
| Method | Links | |
|---|---|---|
| Proprietary API SOTA (SenseTime, 2024)Model Type=Proprietary API, Evaluation Protocol=MCQ2025.01 | 72.7 | |
| Open-Source SOTA (Chen et al., 2024d)Model Type=Open-Source (<10B), Evaluation Protocol=MCQ2025.01 | 63.2 | |
| IXC-2.5Model Type=Open-Source (<10B), Evaluation Protocol=MCQ2025.01 | 59.9 | |
| IXC-2.5-ChatModel Type=Open-Source (<10B), Evaluation Protocol=MCQ2025.01 | 59.6 | |
| PivotMergePartitioning=Clustering2026.04 | 27.5 | |
| TSV-MPartitioning=Clustering2026.04 | 27.3 | |
| CC12M Split 5Partitioning=Clustering2026.04 | 26.7 | |
| CC12M Split 3Partitioning=Clustering2026.04 | 26.5 | |
| CC12M Split 2Partitioning=Clustering2026.04 | 25.7 | |
| Task ArithmeticPartitioning=Clustering2026.04 | 25.7 | |
| TIES MergingPartitioning=Clustering2026.04 | 25.1 | |
| Weight AveragePartitioning=Clustering2026.04 | 23.9 | |
| TIES w/ DAREPartitioning=Clustering2026.04 | 23.7 | |
| MetaGPTPartitioning=Clustering2026.04 | 23.3 | |
| CC12M Split 4Partitioning=Clustering2026.04 | 22.9 | |
| CC12M Split 1Partitioning=Clustering2026.04 | 20 | |
| Iso-CPartitioning=Clustering2026.04 | 13.5 |