Multimodal Preference Evaluation on MMRB2
80.2T2I AccuracyMJ1
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MJ1Model Family/Base Model=Qwen3-VL-30B-A3B, Fine-tuning Protocol (e.g., LoRA)=LoRA, Active Parameters=3B2026.03 | 80.2 | 78.1 | 73.5 | 76.4 | 77 | |
| Gemini 3 ProModel Category (Open-source vs API-based)=API-based2026.03 | 74.4 | 74.9 | 76.4 | 79.5 | 76.3 | |
| GPT-5Model Category (Open-source vs API-based)=API-based2026.03 | 70.5 | 73.8 | 74.4 | 70.2 | 72.2 | |
| Gemini 2.5 ProModel Category (Open-source vs API-based)=API-based2026.03 | 70.5 | 71.3 | 75.1 | 66.6 | 70.9 | |
| GPT-4.1Model Category (Open-source vs API-based)=API-based2026.03 | 65.8 | 68.2 | 67 | 53 | 63.5 | |
| Qwen3-VL-32BModel Category (Open-source vs API-based)=Open-source, Active Parameters=32B2026.03 | 64.1 | 67.3 | 70.5 | 56.6 | 64.6 | |
| Gemini 2.5 FlashModel Category (Open-source vs API-based)=API-based2026.03 | 63.1 | 66.5 | 69.4 | 57.5 | 64.1 | |
| Qwen3-VL-235B-A22BModel Category (Open-source vs API-based)=Open-source, Active Parameters=22B2026.03 | 62 | 64.8 | 69 | 55.9 | 62.9 | |
| GPT-4oModel Category (Open-source vs API-based)=API-based2026.03 | 60.3 | 65 | 61.5 | 51.9 | 59.7 | |
| Qwen3-VL-30B-A3BModel Category (Open-source vs API-based)=Open-source, Active Parameters=3B2026.03 | 60 | 59.5 | 57.3 | 57.3 | 58.5 | |
| Qwen3-VL-8BModel Category (Open-source vs API-based)=Open-source, Active Parameters=8B2026.03 | 59.4 | 61.7 | 61.5 | 54.6 | 59.3 | |
| Qwen2.5-VL-72BModel Category (Open-source vs API-based)=Open-source, Active Parameters=72B2026.03 | 59.1 | 64.6 | 62.3 | 50 | 59 | |
| Gemma 3 27BModel Category (Open-source vs API-based)=Open-source, Active Parameters=27B2026.03 | 58.3 | 60.2 | 61.1 | 49.4 | 57.3 | |
| Gemma 3 12BModel Category (Open-source vs API-based)=Open-source, Active Parameters=12B2026.03 | 56 | 58 | 58 | 49.3 | 55.3 | |
| Gemma 3 4BModel Category (Open-source vs API-based)=Open-source, Active Parameters=4B2026.03 | 51.7 | 51 | 51.3 | 48.8 | 50.7 | |
| Qwen2.5-VL-7BModel Category (Open-source vs API-based)=Open-source, Active Parameters=7B2026.03 | 50.4 | 57.1 | 48.4 | 47.5 | 50.9 |