Preference Prediction on MMRB2 out-of-domain
70.7EvalMuse ScoreAutoRubric-T2I (on PickScore)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| AutoRubric-T2I (on PickScore)Base Model=Gemini-3-Flash, Judging Protocol=Pointwise2026.05 | 70.7 | 71.9 | 71.1 | 79.5 | 66 | 71.4 | — | |
| AutoRubric-T2I (on HPSv3)Base Model=Gemini-3-Flash, Judging Protocol=Pointwise2026.05 | 70.2 | 71.3 | 70.8 | 78.7 | 64.9 | 70.8 | — | |
| Gemini-3-FlashJudging Protocol=Pairwise2026.05 | 69.8 | 73.2 | 70.7 | 78.5 | 65 | 70.8 | — | |
| AutoRubric-T2I (on PickScore)Base Model=Qwen3-VL-32B, Judging Protocol=Pointwise2026.05 | 66.9 | 68.8 | 67.8 | 73.4 | 62.4 | 67.7 | — | |
| AutoRubric-T2I (on HPSv3)Base Model=Qwen3-VL-32B, Judging Protocol=Pointwise2026.05 | 64.2 | 67.3 | 68.4 | 69.1 | 64.5 | 65.4 | — | |
| Qwen3-VL-32BJudging Protocol=Pairwise2026.05 | 63.3 | 68.3 | 60.9 | 65.6 | 59 | 64.1 | — | |
| AutoRubric-T2I (on PickScore)Base Model=Qwen3-VL-8B, Judging Protocol=Pointwise2026.05 | 60.8 | 66.2 | 65.6 | 61.3 | 55.9 | 62.4 | — | |
| Qwen3-VL-8BJudging Protocol=Pairwise2026.05 | 59.2 | 62.8 | 57.4 | 62.4 | 50.9 | 59.4 | — | |
| AutoRubric-T2I (on HPSv3)Base Model=Qwen3-VL-8B, Judging Protocol=Pointwise2026.05 | 58.5 | 67.3 | 64.1 | 65.6 | 60.4 | 62.5 | — | |
| PickScoreCategory=CLIP-Based2026.05 | 57.2 | 54.3 | 67.2 | 53.8 | 57.7 | 57.4 | — | |
| UnifiedRewardBase Model=Qwen2.5-VL-7B, Category=Fine-Tuned2026.05 | 56.9 | 62.1 | 56.8 | 67.8 | 56 | 59.8 | — | |
| AutoRule (on HPSv3)Base Model=Qwen3-VL-8B, Judging Protocol=Pointwise2026.05 | 56.9 | 60.4 | 64.1 | 62.4 | 55 | 59.1 | — | |
| CLIPScoreCategory=CLIP-Based2026.05 | 54.4 | 51.1 | 53.1 | 46.2 | 55 | 52.6 | — | |
| AutoRubric (on HPSv3)Base Model=Qwen3-VL-8B, Judging Protocol=Pointwise2026.05 | 54.1 | 61.9 | 61.7 | 59.1 | 52.3 | 57.5 | — | |
| HPSv3Base Model=Qwen2.5-VL-7B, Category=Fine-Tuned2026.05 | 54 | 60.4 | 68 | 68.9 | 56.7 | 59.4 | — | |
| AutoRubric (on PickScore)Base Model=Qwen3-VL-8B, Judging Protocol=Pointwise2026.05 | 53.1 | 56.1 | 59.4 | 59.1 | 68.5 | 57 | — | |
| ImageRewardCategory=CLIP-Based2026.05 | 51 | 51.4 | 58.6 | 52.7 | 57.7 | 53 | — | |
| HPSv2Category=CLIP-Based2026.05 | 51 | 53.6 | 63.3 | 60.2 | 55 | 54.6 | — | |
| AutoRule (on PickScore)Base Model=Qwen3-VL-8B, Judging Protocol=Pointwise2026.05 | 50.8 | 61.5 | 60.2 | 58.1 | 57.7 | 56.4 | — | |
| Gemini-3-FlashJudging Protocol=Pointwise2026.05 | 36.7 | 38.4 | 38.6 | 34.1 | 30.9 | 36.6 | 62.1 | |
| Qwen3-VL-8BJudging Protocol=Pointwise2026.05 | 29.8 | 27 | 25.8 | 23.1 | 17.7 | 26.5 | 58 | |
| Qwen3-VL-32BJudging Protocol=Pointwise2026.05 | 25.9 | 21.6 | 20.3 | 21.5 | 15.3 | 22.4 | 56.9 |