Multi-modal Evaluation on DEMON Benchmark (zero-shot)
37.5Multi Modal DialogueVPG-C
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| VPG-Cevaluation_mode=zero-shot2024.08 | 37.5 | 25.2 | 25.9 | 22.15 | 48.6 | 44.93 | 50.28 | |
| MaVEnevaluation_mode=zero-shot2024.08 | 34.63 | 21.53 | 30.24 | 33.35 | 51.53 | 47.33 | 54.38 | |
| InstructBLIPevaluation_mode=zero-shot2024.08 | 33.58 | 24.41 | 11.49 | 21.2 | 47.4 | 44.4 | 48.55 | |
| LLaVA-1.5evaluation_mode=zero-shot2024.08 | 27.17 | 14.32 | 11.62 | 31.65 | 46.4 | 38.87 | 44.58 | |
| OpenFlamingoevaluation_mode=zero-shot2024.08 | 16.88 | 24.22 | 13.85 | 21.65 | 32 | 30.6 | 41.63 | |
| Otterevaluation_mode=zero-shot2024.08 | 15.37 | 15.57 | 11.39 | 16 | 41.67 | 27.73 | 43.85 | |
| LLaMA-Adapter-v2evaluation_mode=zero-shot2024.08 | 14.22 | 17.57 | 13.51 | 18 | 44.8 | 32 | 44.03 | |
| MiniGPT-4evaluation_mode=zero-shot2024.08 | 13.7 | 17.07 | 7.95 | 16.6 | 30.27 | 26.4 | 43.5 | |
| mPLUG-Owlevaluation_mode=zero-shot2024.08 | 12.67 | 19.33 | 5.4 | 16.25 | 33.27 | 32.47 | 42.5 | |
| BLIP-2evaluation_mode=zero-shot2024.08 | 11.96 | 20.1 | 3.67 | 18.25 | 39.73 | 30.53 | 39.53 | |
| LLaVAevaluation_mode=zero-shot2024.08 | 7.79 | 10.7 | 8.27 | 15.85 | 36.2 | 28.33 | 41.53 |