Multi-modal Hallucination Evaluation on MMHal-Bench v1.0 (test)
2.14Overall ScoreInstructBLIP
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| InstructBLIPSize=13B2023.12 | 2.14 | 58 | 2.75 | 1.75 | 1.25 | 2.08 | 2.5 | 4.08 | 1.5 | 1.17 | |
| LLaVA1.5-HACLSize=7B, method=HACL2023.12 | 2.13 | 50 | 2.95 | 2.15 | 2.29 | 1.97 | 1.53 | 1.98 | 2.02 | 2.19 | |
| InstructBLIPSize=7B2023.12 | 2.1 | 58 | 3.42 | 2.08 | 1.33 | 1.92 | 2.17 | 3.67 | 1.17 | 1.08 | |
| LLaVA-HACLSize=7B, method=HACL2023.12 | 2.08 | 62 | 2.94 | 2.01 | 2.27 | 1.64 | 2.35 | 2.14 | 1.67 | 1.63 | |
| LLAVA1.5Size=7B2023.12 | 2.08 | 52 | 2.75 | 2 | 2.33 | 2.08 | 1.5 | 1.91 | 1.91 | 2.16 | |
| IDEFICSSize=80B2023.12 | 2.05 | 61 | 2.33 | 1.25 | 2 | 2.5 | 1.5 | 3.33 | 2.33 | 1.17 | |
| LLAVA-RLHFSize=7B2023.12 | 2.05 | 68 | 2.92 | 1.83 | 2.42 | 1.92 | 2.25 | 2.25 | 1.75 | 1.08 | |
| IDEFICSSize=9B2023.12 | 1.89 | 64 | 1.58 | 0.75 | 2.75 | 1.83 | 1.83 | 2.5 | 2.17 | 1.67 | |
| miniGPT-4-HACLSize=7B, method=HACL2023.12 | 1.8 | 65 | 1.22 | 1.85 | 2.23 | 1.74 | 2.13 | 2.48 | 1.03 | 1.58 | |
| Kosmos-22023.12 | 1.69 | 68 | 2 | 0.25 | 1.42 | 1.67 | 1.67 | 2.67 | 2.5 | 1.33 | |
| LLaVASize=7B2023.12 | 1.55 | 76 | 1.33 | 0 | 1.83 | 1.17 | 2 | 2.58 | 1.67 | 1.83 | |
| miniGPT-4Size=7B2023.12 | 1.39 | 71 | 0.75 | 1.83 | 2.16 | 0.91 | 1.25 | 1.33 | 0.91 | 1.91 |