Relation Hallucination Evaluation on Reefknot 1.0 (full)
46.7Perception Y/N AccuracyMiniGPT4-v2
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MiniGPT4-v2Size=7B, Temperature=02024.08 | 46.7 | 78 | 61.3 | 43.73 | 68.5 | 65.88 | 39.88 | |
| Qwen-VLSize=7B, Temperature=02024.08 | 42.21 | 56.7 | 72.47 | 33.53 | 21.88 | 73.01 | 52.55 | |
| LLaVASize=13B, Temperature=02024.08 | 40.7 | 59.35 | 48.93 | 34.19 | 29.19 | 54.45 | 57.47 | |
| Phi-3-vision-128k-instructSize=4.2B, Temperature=02024.08 | 39.88 | 57.07 | 50.98 | 33.97 | 21.35 | 49.45 | 60.3 | |
| LLaVASize=7B, Temperature=02024.08 | 37.67 | 68.05 | 52.93 | 33.99 | 51.04 | 54.56 | 51.41 | |
| Deepseek-VLSize=7B, Temperature=02024.08 | 37.58 | 56.33 | 67.07 | 32.22 | 23.6 | 59.34 | 56.39 | |
| CogVLMSize=19B, Temperature=02024.08 | 37.23 | 47.95 | 70.14 | 29.89 | 18.54 | 66.18 | 57.1 | |
| GLM4VSize=9B, Temperature=02024.08 | 34.09 | 50.47 | 58.09 | 27.08 | 16.87 | 56.47 | 62.03 | |
| Yi-VLSize=6B, Temperature=02024.08 | 33.56 | 47.53 | 71.02 | 33.16 | 16.33 | 74.96 | 55.81 | |
| Yi-VLSize=34B, Temperature=02024.08 | 32.79 | 44.19 | 57.67 | 33.75 | 14.85 | 52.72 | 62.61 | |
| GPT-4oTemperature=02024.08 | 32.56 | 40.93 | 42.7 | 26.27 | 11.53 | 48.78 | 68.32 | |
| MiniCPMSize=7B, Temperature=02024.08 | 31.93 | 48.65 | 47.63 | 27.65 | 16.71 | 45.96 | 65.73 |