Human Consistency Evaluation on MLLM-as-a-Judge
30.3CO Consistency ScoreLLaVA-Critic
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaVA-CriticModel Category=Open-source, Parameters=72B2025.06 | 30.3 | 33.8 | 2.9 | 34.8 | 38.1 | 37.4 | 13.5 | 43.1 | 34.8 | 20.4 | 7.6 | 25.6 | 17 | 29.3 | |
| Gemini-2.5-ProModel Category=Closed-source2025.06 | 29.9 | 34.1 | 36 | 48.1 | 42.1 | 47.4 | 28.6 | 51.2 | 40.5 | 18.9 | -0.2 | 34.5 | 18.8 | 23.2 | |
| UnifiedReward_QModel Category=Open-source, Parameters=8B2025.06 | 26.9 | 30.3 | 17.6 | 37.7 | 37.8 | 35.4 | 29.5 | 48.7 | 29.2 | 21.4 | 7.9 | 34.8 | 20.9 | 28.3 | |
| MinosModel Category=Open-source, Parameters=8B2025.06 | 24.9 | 35.7 | 23.1 | 48.5 | 41.4 | 44.5 | 36.6 | 50.9 | 35 | 25.2 | 12 | 38.7 | 23.8 | 25.8 | |
| UnifiedReward_LModel Category=Open-source, Parameters=7B2025.06 | 23.7 | 31.9 | 13.8 | 29.5 | 34.2 | 35 | 20.2 | 35.7 | 30 | 18.3 | 12.5 | 31.4 | 27 | 14.9 | |
| GPT-4oModel Category=Closed-source2025.06 | 23.1 | 28.8 | 16.1 | 36.8 | 35.9 | 39.6 | 30.3 | 47.2 | 31.9 | 18.3 | 1.7 | 28.8 | 20.4 | 29 | |
| LLaVA-OVModel Category=Open-source, Parameters=72B2025.06 | 20.2 | 13.3 | 7.5 | 14.2 | 24.1 | 22.2 | 4.6 | 19.7 | 27.2 | 21.9 | 19 | 13.8 | 19.6 | 26.9 | |
| LLaVA-CriticModel Category=Open-source, Parameters=7B2025.06 | 19.2 | 31.6 | 10.3 | 30.1 | 23.3 | 29.1 | 8.3 | 25 | 25.4 | 15.6 | 14.8 | 7.7 | 28.6 | 25.3 | |
| Qwen3-VLModel Category=Open-source, Parameters=8B2025.06 | 17.3 | 31.2 | 23.6 | 49.1 | 40.9 | 41 | 37.9 | 49.3 | 36.4 | 22 | -0.7 | 31.7 | 22.9 | 22.6 | |
| LLaVA-OVModel Category=Open-source, Parameters=7B2025.06 | 15.3 | 16.9 | 10.4 | 12.9 | 11.5 | 19.8 | 3.3 | 28.2 | 19.6 | 15.9 | 17.5 | 6.6 | 26.9 | 26 | |
| Prometheus-VModel Category=Open-source, Parameters=7B2025.06 | 11.2 | 9.1 | 9.4 | 12.7 | 11.1 | 17.8 | 18 | 8.4 | 11.5 | 15.5 | 11.4 | 8.2 | 24.1 | 11 |