Visual Question Answering on Vision-Language Evaluation Suite (test)
69.56MMB ScoreTop-6
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Top-6Backbone=DeepSeek-VL2-Tiny-1B-3B, N_A=1.17B, Avg_k=6.002026.06 | 69.56 | 89.07 | 88.99 | 70.16 | 60.84 | 49.1 | 71.29 | |
| Top-5Backbone=DeepSeek-VL2-Tiny-1B-3B, N_A=1.13B, Avg_k=5.002026.06 | 69.39 | 88.9 | 88.55 | 69.82 | 59.76 | 48.6 | 70.84 | |
| Top-4Backbone=DeepSeek-VL2-Tiny-1B-3B, N_A=1.09B, Avg_k=4.002026.06 | 68.87 | 88.7 | 87.24 | 68.72 | 59.63 | 48.5 | 70.28 | |
| GeMoEBackbone=DeepSeek-VL2-Tiny-1B-3B, N_A=1.08B, Avg_k=3.552026.06 | 68.72 | 89.41 | 88.78 | 69.24 | 60.37 | 48.89 | 70.9 | |
| Top-3Backbone=DeepSeek-VL2-Tiny-1B-3B, N_A=1.06B, Avg_k=3.002026.06 | 67.75 | 88.39 | 87.1 | 67.75 | 59.12 | 47.1 | 69.54 | |
| Top-8Backbone=MolmoE-1B-7B, N_A=1.58B, Avg_k=8.002026.06 | 58.24 | 85.33 | 87.5 | 56.37 | 50.89 | 45.6 | 63.99 | |
| Top-7Backbone=MolmoE-1B-7B, N_A=1.48B, Avg_k=7.002026.06 | 57.3 | 84.79 | 86.51 | 56.07 | 50.76 | 43.9 | 63.22 | |
| GeMoEBackbone=MolmoE-1B-7B, N_A=1.32B, Avg_k=5.432026.06 | 56.7 | 88.38 | 86.98 | 54.88 | 50.64 | 44.1 | 63.61 | |
| Top-6Backbone=MolmoE-1B-7B, N_A=1.38B, Avg_k=6.002026.06 | 53.86 | 84.55 | 85.72 | 55.34 | 50.21 | 43.4 | 62.18 | |
| Top-5Backbone=MolmoE-1B-7B, N_A=1.28B, Avg_k=5.002026.06 | 48.11 | 84.43 | 84.09 | 51.95 | 48.96 | 42.9 | 60.07 |