Spatial-Conditioned Reasoning on SCaR
64.2RefCOCO+ ScoreVIRTUE-2B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VIRTUE-2Bfinetuning_on_MMEB=MMEB-Train, additional_finetuning=SCaR-train2025.10 | 64.2 | 65.3 | 41.4 | 54.2 | 56 | 56.2 | |
| VIRTUE-7Bfinetuning_on_MMEB=MMEB-Train, additional_finetuning=SCaR-train2025.10 | 63.2 | 66.1 | 48.2 | 52.2 | 54.9 | 56.9 | |
| VLM2Vec-2Bfinetuning_on_MMEB=MMEB-Train, additional_finetuning=SCaR-train2025.10 | 59.8 | 55.5 | 34.1 | 40.8 | 43.2 | 46.7 | |
| UniME-7Bfinetuning_on_MMEB=MMEB-Train, additional_finetuning=SCaR-train2025.10 | 57.8 | 59.3 | 44.5 | 41.9 | 43.6 | 49.4 | |
| MMRet-7Bfinetuning_on_MMEB=MMEB-Train, additional_finetuning=SCaR-train2025.10 | 45.8 | 43.6 | 39.7 | 27.5 | 31.2 | 37.6 | |
| VLM2Vec-7Bfinetuning_on_MMEB=MMEB-Train, additional_finetuning=SCaR-train2025.10 | 40.1 | 39.5 | 36.3 | 31.4 | 25 | 34.5 | |
| VIRTUE-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=standard2025.10 | 33 | 35.3 | 19.6 | 27.1 | 23.8 | 27.8 | |
| UniME-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=standard2025.10 | 31.4 | 32.8 | 19 | 25.3 | 23 | 26.3 | |
| VIRTUE-2Bfinetuning_on_MMEB=MMEB-Train, input_processing=standard2025.10 | 28.8 | 42.4 | 24.4 | 29.9 | 27.5 | 30.4 | |
| MMRet-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=standard2025.10 | 27.1 | 21.8 | 15.2 | 26 | 22.6 | 22.5 | |
| VIRTUE-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=Cropping2025.10 | 27.1 | 31.5 | 17.4 | 21.8 | 16.4 | 22.8 | |
| MMRet-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=Cropping2025.10 | 26 | 20.2 | 14.9 | 15 | 19.1 | 19 | |
| UniME-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=Cropping2025.10 | 25.8 | 24 | 15.2 | 24.6 | 22.3 | 22.4 | |
| VLM2Vec-2Bfinetuning_on_MMEB=MMEB-Train, input_processing=standard2025.10 | 24.5 | 29.5 | 22.3 | 19.4 | 24.6 | 24.1 | |
| VIRTUE-2Bfinetuning_on_MMEB=MMEB-Train, input_processing=Cropping2025.10 | 24.4 | 36.3 | 14.2 | 25.8 | 23.4 | 24.8 | |
| VLM2Vec-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=standard2025.10 | 23.2 | 29.1 | 14.7 | 25 | 22.4 | 22.9 | |
| VLM2Vec-2Bfinetuning_on_MMEB=MMEB-Train, input_processing=Cropping2025.10 | 22.3 | 25.8 | 17.1 | 19.5 | 22.5 | 21.4 | |
| CLIPfinetuning_on_MMEB=none, input_processing=standard2025.10 | 18.1 | 22.4 | 23 | 15.5 | 18.9 | 19.6 | |
| ReCLIPfinetuning_on_MMEB=none, input_processing=standard2025.10 | 17.6 | 23 | 21.3 | 12.6 | 15.9 | 18.1 | |
| ReCLIPfinetuning_on_MMEB=none, input_processing=Red Circle2025.10 | 17.6 | 21.6 | 22.3 | 15.1 | 18.4 | 19 | |
| VLM2Vec-7Bfinetuning_on_MMEB=MMEB-Train, input_processing=Cropping2025.10 | 17.6 | 25.7 | 15.2 | 13.8 | 21.3 | 18.7 | |
| LamRA-7Bfinetuning_on_MMEB=none, input_processing=Cropping2025.10 | 8.2 | 7.9 | 3.1 | 8.9 | 5.3 | 6.7 | |
| LamRA-7Bfinetuning_on_MMEB=none, input_processing=standard2025.10 | 8 | 7.9 | 5.3 | 3.1 | 8.9 | 6.6 | |
| GME-2Bfinetuning_on_MMEB=none, input_processing=Cropping2025.10 | 6.1 | 8.3 | 5.8 | 6.2 | 7.5 | 6.8 | |
| GME-7Bfinetuning_on_MMEB=none, input_processing=standard2025.10 | 6.1 | 10.1 | 7.4 | 7 | 5.6 | 7.2 | |
| GME-7Bfinetuning_on_MMEB=none, input_processing=Cropping2025.10 | 6 | 9.8 | 4.4 | 7 | 5.3 | 6.5 | |
| GME-2Bfinetuning_on_MMEB=none, input_processing=standard2025.10 | 5.4 | 8.3 | 7.5 | 5.8 | 6.2 | 6.6 |