Multimodal Understanding on DEMON
51.8MMDSugar
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| SugarLLM=Vicuna-7B2024.11 | 51.8 | 34.3 | 32.3 | 16.8 | 64.4 | 65.9 | 51.7 | |
| VILA-7BLLM=Vicuna-7B2024.11 | 47.8 | 25.8 | 13.2 | 17.2 | 60.1 | 42.1 | 50.5 | |
| VPG-CLLM=Vicuna-7B2024.11 | 37.5 | 25.2 | 25.9 | 22.2 | 48.6 | 44.9 | 50.3 | |
| InstructBLIPLLM=Vicuna-7B2024.11 | 33.6 | 24.4 | 11.5 | 21.2 | 47.4 | 44.4 | 48.6 | |
| BLIP-2LLM=Vicuna-13B2024.11 | 26.1 | 21.3 | 10.7 | 17.9 | 39.2 | 33.5 | 39.7 | |
| OpenFlamingoLLM=MPT-7B2024.11 | 16.9 | 24.2 | 13.9 | 21.7 | 32 | 30.6 | 41.6 | |
| MiniGPT-4LLM=Vicuna-7B2024.11 | 13.7 | 17.1 | 8 | 16.6 | 30.3 | 26.4 | 43.5 | |
| mPlug-OwlLLM=LLaMA-7B2024.11 | 12.7 | 19.3 | 5.4 | 16.3 | 33.3 | 32.5 | 42.5 | |
| LLaVALLM=Vicuna-7B2024.11 | 7.8 | 10.7 | 8.3 | 15.9 | 36.2 | 28.3 | 41.5 |