Visual Question Answering on MathVerse vision-only (test)
54.8AccuracyProprietary API SOTA (Google, 2024)
Evaluation Results
| Method | Links | |
|---|---|---|
| Proprietary API SOTA (Google, 2024)Model Type=Proprietary API, Evaluation Protocol=VQA2025.01 | 54.8 | |
| Open-Source SOTA (Liu et al., 2024c)Model Type=Open-Source (<10B), Evaluation Protocol=VQA2025.01 | 26.6 | |
| IXC-2.5-ChatModel Type=Open-Source (<10B), Evaluation Protocol=VQA2025.01 | 19 | |
| IXC-2.5Model Type=Open-Source (<10B), Evaluation Protocol=VQA2025.01 | 16.2 |