Prediction Reasoning on MMBench (dev)
64.4Avg. AccuracyMerlin
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MerlinLLM Size=7B2023.11 | 64.4 | 42 | 54.7 | 72.8 | 97.8 | 54.8 | |
| LLaVA-1.5LLM Size=7B2023.11 | 59.6 | 43.2 | 52 | 71.6 | 93.3 | 38.1 | |
| Kosmos-2LLM Size=1.6B2023.11 | 54.4 | 38.3 | 33.3 | 56.8 | 91.1 | 52.4 | |
| ShikraLLM Size=7B2023.11 | 51.5 | 32.1 | 30.7 | 63 | 88.9 | 42.9 | |
| MiniGPT-4LLM Size=13B2023.11 | 43.3 | 28.4 | 30.7 | 49.4 | 86.7 | 21.4 | |
| InstructBLIPLLM Size=13B2023.11 | 42 | 14.8 | 30.7 | 56.8 | 88.9 | 19 | |
| mPLUG-OwlLLM Size=7B2023.11 | 41 | 18.5 | 18.7 | 66.7 | 86.7 | 14.3 | |
| LLAVALLM Size=7B2023.11 | 38.7 | 8.6 | 25.3 | 53.1 | 77.8 | 28.6 | |
| InstructBLIPLLM Size=7B2023.11 | 34.8 | 6.2 | 17.3 | 51.9 | 84.4 | 14.3 | |
| MiniGPT-4LLM Size=7B2023.11 | 26.8 | 7.4 | 14.7 | 19.8 | 80 | 11.9 | |
| MMGPTLLM Size=7B2023.11 | 19.5 | 1.2 | 24 | 9.9 | 60 | 2.4 | |
| OpenFlamingoLLM Size=7B2023.11 | 5.28 | 2.5 | 10.7 | 8.6 | 2.2 | 2.4 |