Multimodal Evaluation Consistency on MLLM-as-a-Judge
39.6CO ScoreGPT-4o
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4oScale=/2025.06 | 39.6 | 45.2 | 34.1 | 46.4 | 46 | 56.4 | 40.8 | 57.3 | 58.9 | 30.5 | 26.2 | 56.9 | 42.1 | 34.2 | — | |
| Gemini-2.5-ProScale=/2025.06 | 38.9 | 40.7 | 43.4 | 56.8 | 49.5 | 59.6 | 34 | 60.7 | 50.1 | 22.7 | -0.5 | 40 | 20 | 37.3 | — | |
| Gemini-2.5-ProModel=Gemini-2.5-Pro2025.06 | 38.9 | 40.7 | 43.4 | 56.8 | 49.5 | 59.6 | 34 | 60.7 | 50.1 | 22.7 | -0.5 | 40 | 20 | 37.3 | — | |
| LLaVA-CriticScale=7B2025.06 | 38.2 | 45 | 10.3 | 31.6 | 35.6 | 37.8 | 17.9 | 42.1 | 32.2 | 24.6 | 30.1 | 26.9 | 39.5 | 27.2 | — | |
| LLaVA-Critic(72B)Model=LLaVA-Critic(72B)2025.06 | 37 | 36 | 8.6 | 43.1 | 45.7 | 46.8 | 20.7 | 53 | 44 | 26.5 | 14.7 | 35.2 | 27.1 | 42.2 | — | |
| LLaVA-CriticScale=72B2025.06 | 33.3 | 46.3 | 14.6 | 45.2 | 47.4 | 55.9 | 39.6 | 54.5 | 48.8 | 27.3 | 25.9 | 33.4 | 40.3 | 37.4 | — | |
| MinosScale=8B2025.06 | 32.8 | 41.1 | 25.1 | 54.1 | 51.3 | 58.3 | 41.4 | 62.1 | 47.1 | 24.3 | 23.6 | 50.7 | 28.7 | 39.8 | — | |
| Minos(8B)Model=Minos(8B)2025.06 | 32.8 | 41.1 | 25.1 | 54.1 | 51.3 | 58.3 | 41.4 | 62.1 | 47.1 | 24.3 | 23.6 | 50.7 | 28.7 | 39.8 | — | |
| GPT-4oModel=GPT-4o2025.06 | 30.1 | 33 | 22.2 | 44.3 | 44.3 | 52.8 | 32.8 | 57.4 | 40.1 | 22.4 | 6.5 | 36.2 | 29.3 | 31.2 | — | |
| UnifiedReward_QScale=8B2025.06 | 29.3 | 35.1 | 22.3 | 44 | 46.6 | 43.4 | 29 | 55.3 | 37.9 | 25 | 17.2 | 43.1 | 30.6 | 33.6 | — | |
| UnifiedReward_Q(8B)Model=UnifiedReward_Q(8B)2025.06 | 29.3 | 35.1 | 22.3 | 44 | 46.6 | 43.4 | 29 | 55.3 | 37.9 | 25 | 17.2 | 43.1 | 30.6 | 33.6 | — | |
| Prometheus-VScale=7B2025.06 | 28.9 | 34.2 | 10.6 | 17.2 | 18.2 | 21.4 | 20.9 | 22.4 | 22.6 | 22.8 | 8.9 | 17.4 | 36.8 | 15.7 | — | |
| LLaVA-OV(72B)Model=LLaVA-OV(72B)2025.06 | 27.5 | 12.4 | 9.7 | 20.1 | 31.6 | 31.3 | 8.9 | 26 | 35.1 | 25.6 | 31.5 | 27.2 | 26.3 | 37.2 | — | |
| LLaVA-OVScale=72B2025.06 | 26.4 | 39 | 4.6 | 26.2 | 35.8 | 32.7 | 19.5 | 29 | 41.5 | 14.4 | 35.9 | 26.7 | 44.4 | 25.3 | — | |
| Qwen3-VLScale=8B2025.06 | 26.4 | 37.3 | 28.5 | 57 | 49 | 51.2 | 43.2 | 56.4 | 43.3 | 27.2 | -0.1 | 39.6 | 24.8 | 29.3 | — | |
| Qwen3-VL(8B)Model=Qwen3-VL(8B)2025.06 | 26.4 | 37.3 | 28.5 | 57 | 49 | 51.2 | 43.2 | 56.4 | 43.3 | 27.2 | -0.1 | 39.6 | 24.8 | 29.3 | — | |
| LLaVA-Critic(7B)Model=LLaVA-Critic(7B)2025.06 | 25.9 | 34.3 | 15.1 | 37.9 | 28.5 | 38.4 | 17.6 | 32 | 34.1 | 20.1 | 23.1 | 15.6 | 39.2 | 32.3 | — | |
| UnifiedReward_LScale=7B2025.06 | 25 | 37.8 | 19.6 | 34.4 | 40.5 | 44 | 22.6 | 39.5 | 35.9 | 22.9 | 22 | 36.9 | 32.4 | 21.8 | — | |
| UnifiedReward_L(7B)Model=UnifiedReward_L(7B)2025.06 | 25 | 37.8 | 19.6 | 34.4 | 40.5 | 44 | 22.6 | 39.5 | 35.9 | 22.9 | 22 | 36.9 | 32.4 | 21.8 | — | |
| LLaVA-OVScale=7B2025.06 | 22.4 | 2.4 | 6.3 | 18.9 | 9.7 | 26.5 | -13.5 | 27.4 | 22.7 | 8.1 | 3 | 26.1 | 24.9 | 26.2 | — | |
| LLaVA-OV(7B)Model=LLaVA-OV(7B)2025.06 | 20.6 | 19.4 | 13.6 | 21.1 | 14.8 | 28.2 | 4.9 | 30.3 | 29.6 | 16.9 | 34.2 | 13.5 | 42.3 | 39.1 | — | |
| Prometheus-V(7B)Model=Prometheus-V(7B)2025.06 | 13.4 | 10.3 | 12.2 | 15.4 | 13.7 | 21.3 | 19.7 | 13.5 | 14.2 | 16.1 | 18.1 | 11.9 | 36.3 | 15.8 | — |