Preference Evaluation on Detail
8.25Avg ScorePluralistic
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PluralisticReward Model Setting=Homogeneous2026.01 | 8.25 | 90.6 | 7.53 | |
| Ours (Homo.)Reward Model Setting=Homogeneous2026.01 | 8.15 | 89.72 | 7.4 | |
| FedAvgReward Model Setting=Homogeneous2026.01 | 8.08 | 83.7 | 7.34 | |
| RM1Reward Model Setting=Homogeneous2026.01 | 8 | 85.32 | 7.21 | |
| Random SelectionReward Model Setting=Homogeneous2026.01 | 7.93 | 83.06 | 7.08 | |
| Ours (Hete.)Reward Model Setting=Heterogeneous2026.01 | 7.73 | 80.88 | 6.8 | |
| Random SelectionReward Model Setting=Heterogeneous2026.01 | 7.6 | 77.39 | 6.57 | |
| Oracle RMReward Model Setting=Homogeneous2026.01 | 7.4 | 74.89 | 6.23 | |
| RM0Reward Model Setting=Heterogeneous, Backbone=Qwen2-VL-2B-Instruct2026.01 | 7.11 | 73.57 | 5.98 | |
| RM2Reward Model Setting=Heterogeneous, Backbone=Qwen2.5-VL-3B-Instruct2026.01 | 7.01 | 80.2 | 5.73 | |
| RM2Reward Model Setting=Homogeneous2026.01 | 7.01 | 80.2 | 5.73 | |
| RM0Reward Model Setting=Homogeneous2026.01 | 6.12 | 61.09 | 5.59 | |
| Avg RMReward Model Setting=Heterogeneous2026.01 | 4.97 | 60 | 3.26 | |
| RM1Reward Model Setting=Heterogeneous, Backbone=llava-onevision-0.5b2026.01 | 3.44 | 33.26 | 2.46 |