Visual Question Answering (Perception + Reasoning) on MathVista, MMStar, and MMVet
46.9MathVista ScoreLATTE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LATTELanguage / Vision=Qwen2-7B/SigLIP, Starting checkpoint=LLaVA-OV Stage 1.52024.12 | 46.9 | 50.8 | 50.9 | 51.2 | |
| Vanilla ITLanguage / Vision=Qwen2-7B/SigLIP, Starting checkpoint=LLaVA-OV Stage 1.52024.12 | 42.4 | 50.1 | 39.3 | 43.9 | |
| LATTELanguage / Vision=LLaMA3-8B/SigLIP, Starting checkpoint=Mantis Instruct-tuned2024.12 | 38.9 | 45.1 | 50 | 44.7 | |
| LATTELanguage / Vision=LLaMA3-8B/CLIP, Starting checkpoint=Mantis Pretrained2024.12 | 36.6 | 40.8 | 45.2 | 40.8 | |
| Vanilla ITLanguage / Vision=LLaMA3-8B/SigLIP, Starting checkpoint=Mantis Instruct-tuned2024.12 | 36.2 | 40.7 | 29.7 | 35.5 | |
| LATTELanguage / Vision=LLaMA3-8B/CLIP, Starting checkpoint=Mantis Pretrained (Configuration 2)2024.12 | 34.9 | 44.6 | 45.2 | 41.6 | |
| Vanilla ITLanguage / Vision=LLaMA3-8B/CLIP, Starting checkpoint=Mantis Pretrained2024.12 | 33.1 | 36.7 | 28.9 | 32.9 | |
| Vanilla ITLanguage / Vision=LLaMA3-8B/CLIP, Starting checkpoint=Mantis Pretrained (Configuration 2)2024.12 | 31.1 | 40.5 | 33 | 34.9 |