Multi-modal Question Answering on MMBench v1.1 (test)
0.857AccuracyProprietary API SOTA (SenseTime, 2024)
Evaluation results
| Method | Links | |
|---|---|---|
| Proprietary API SOTA (SenseTime, 2024)Model Type=Proprietary API, Evaluation Protocol=MCQ2025.01 | 0.857 | |
| Open-Source SOTA (Lu et al., 2024b)Model Type=Open-Source (<10B), Evaluation Protocol=MCQ2025.01 | 0.827 | |
| IXC-2.5Model Type=Open-Source (<10B), Evaluation Protocol=MCQ2025.01 | 0.794 | |
| IXC-2.5-ChatModel Type=Open-Source (<10B), Evaluation Protocol=MCQ2025.01 | 0.79 |