Multimodal Understanding on MME-RealWorld (ID and held-out categories)
84.38Diagram AccuracyMoR
Evaluation Results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MoRConfiguration=3-client homogeneous-RM setting2026.05 | 84.38 | 91.03 | 64.56 | 79.99 | 42.17 | 51.35 | 89.17 | 67.46 | 33.87 | 84.88 | 79.31 | 82.64 | 70.07 | |
| Qwen2.5-VL-7BConfiguration=3-client homogeneous-RM setting2026.05 | 83.75 | 87.18 | 58.27 | 76.4 | 27.71 | 27.03 | 86.62 | 57.14 | 16.94 | 81.4 | 79.77 | 80.99 | 62.44 | |
| w/o OnlineConfiguration=3-client homogeneous-RM setting, Ablation=without online routing2026.05 | 82.5 | 83.97 | 38.58 | 68.35 | 33.73 | 36.04 | 85.35 | 62.7 | 26.61 | 84.88 | 79.77 | 76.86 | 62.82 | |
| FedAvgConfiguration=3-client homogeneous-RM setting2026.05 | 80.63 | 89.1 | 39.37 | 69.7 | 27.71 | 44.14 | 85.99 | 61.11 | 25 | 81.4 | 73.1 | 82.64 | 62.74 | |
| w/o Norm.Configuration=3-client homogeneous-RM setting, Ablation=without normalization2026.05 | 66.25 | 78.21 | 41.73 | 62.06 | 28.92 | 24.32 | 82.17 | 26.98 | 25.81 | 76.74 | 66.44 | 78.51 | 54.19 | |
| PluralLLM-alphaConfiguration=3-client homogeneous-RM setting2026.05 | 63.12 | 75.64 | 21.26 | 53.34 | 24.1 | 22.52 | 64.97 | 34.92 | 33.06 | 66.28 | 60.46 | 63.64 | 48.18 | |
| AvgRMConfiguration=3-client homogeneous-RM setting2026.05 | 62.5 | 77.56 | 48.03 | 62.7 | 37.35 | 59.46 | 84.71 | 59.52 | 24.19 | 80.23 | 63.45 | 75.21 | 61.11 | |
| Best SingleConfiguration=3-client homogeneous-RM setting2026.05 | 60 | 84.62 | 50.39 | 65 | 39.76 | 48.65 | 82.8 | 50.79 | 28.23 | 81.4 | 56.32 | 78.51 | 60.13 | |
| RandomConfiguration=3-client homogeneous-RM setting2026.05 | 50 | 59.62 | 51.97 | 53.86 | 40.96 | 41.44 | 61.15 | 53.97 | 31.45 | 56.98 | 42.76 | 59.5 | 49.98 |