Image Embedding on MMEB v1 (test)
69.1ClassificationUME-R1-7B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| UME-R1-7BMode=Oracle, Model Size/Backbone=7B2025.11 | 69.1 | 73.2 | 74.8 | 87.4 | 79.2 | 67.9 | 74.2 | |
| MMEmb-R1Base Model=Qwen3-VL-4B2026.04 | 67.7 | 74.2 | 74.5 | 94.9 | 75.4 | 74 | 74.8 | |
| mmE5Parameters=11B2026.02 | 67.6 | 62.8 | 70.9 | 89.7 | 72.3 | 66.7 | 69.8 | |
| mmE5-11B2025.11 | 67.6 | 62.8 | 70.9 | 89.7 | 72.3 | 66.7 | 69.8 | |
| UME-R1-2BMode=Oracle, Model Size/Backbone=2B2025.11 | 67.6 | 67.5 | 71.2 | 80.1 | 75.3 | 63.8 | 70.2 | |
| mmE5-11BParameters=11B2026.04 | 67.6 | 62.8 | 70.9 | 89.7 | 72.3 | 66.7 | 69.8 | |
| UME-R1Parameters=7B2026.02 | 67.1 | 69.2 | 71.9 | 84.9 | 76.1 | 65.1 | 71.3 | |
| UME-R1-7BModel Size/Backbone=7B2025.11 | 67.1 | 69.2 | 71.9 | 84.9 | 76.1 | 65.1 | 71.3 | |
| UME-R1-7BParameters=7B2026.04 | 67.1 | 69.2 | 71.9 | 84.9 | 76.1 | 65.1 | 71.3 | |
| UniMEParameters=7B2026.02 | 66.8 | 66.6 | 70.6 | 90.9 | 74.6 | 65.8 | 70.7 | |
| UniME-7B2025.11 | 66.8 | 66.6 | 70.6 | 90.9 | 74.6 | 65.8 | 70.7 | |
| UniME-7BParameters=7B2026.04 | 66.8 | 66.6 | 70.6 | 90.9 | 74.6 | 65.8 | 70.7 | |
| LLaVEParameters=7B2026.02 | 65.7 | 65.4 | 70.9 | 91.9 | 75 | 64.4 | 70.3 | |
| LLaVE-7B2025.11 | 65.7 | 65.4 | 70.9 | 91.9 | 75 | 64.4 | 70.3 | |
| LLaVE-7BParameters=7B2026.04 | 65.7 | 65.4 | 70.9 | 91.9 | 75 | 64.4 | 70.3 | |
| CAFe-V1Parameters=7B2026.02 | 65.2 | 65.6 | 70 | 91.2 | 75.8 | 62.4 | 69.8 | |
| CAFe-V1-7BTraining data=MMEB-V1 training data (contains only image data)2025.11 | 65.2 | 65.6 | 70 | 91.2 | 75.8 | 62.4 | 69.8 | |
| CAFe-V1-7BParameters=7B2026.04 | 65.2 | 65.6 | 70 | 91.2 | 75.8 | 62.4 | 69.8 | |
| UME-R1Parameters=2B2026.02 | 64.8 | 62.8 | 67.6 | 77.2 | 71.5 | 60.4 | 66.6 | |
| UME-R1-2BModel Size/Backbone=2B2025.11 | 64.8 | 62.8 | 67.6 | 77.2 | 71.5 | 60.4 | 66.6 | |
| UME-R1-2BParameters=2B2026.04 | 64.8 | 62.8 | 67.6 | 77.2 | 71.5 | 60.4 | 66.6 | |
| MMEmb-R1Base Model=Qwen2-VL-2B2026.04 | 64.5 | 69 | 70 | 88.9 | 71.4 | 68.4 | 70 | |
| Embed-RLParameters=4B2026.02 | 63.7 | 70.5 | 71.3 | 91.4 | 74.3 | 67.3 | 71.2 | |
| Embed-RL-4BParameters=4B2026.04 | 63.7 | 70.5 | 71.3 | 91.4 | 74.3 | 67.3 | 71.2 | |
| CAFe-V2Parameters=7B2026.02 | 63.6 | 61.7 | 69.1 | 87.6 | 72.8 | 61.1 | 67.6 | |
| CAFe-V2-7BTraining data=MMEB-V2 training data2025.11 | 63.6 | 61.7 | 69.1 | 87.6 | 72.8 | 61.1 | 67.6 | |
| CAFe-V2-7BParameters=7B2026.04 | 63.6 | 61.7 | 69.1 | 87.6 | 72.8 | 61.1 | 67.6 | |
| VLM2Vec-V22026.02 | 62.9 | 56.3 | 69.5 | 77.3 | 68.8 | 59.9 | 64.9 | |
| VLM2Vec-V2Model Size/Backbone=Qwen2-VL-7B2025.11 | 62.9 | 56.3 | 69.5 | 77.3 | 68.8 | 59.9 | 64.9 | |
| VLM2Vec-V22026.04 | 62.9 | 56.3 | 69.5 | 77.3 | 68.8 | 59.9 | 64.9 | |
| Embed-RLParameters=2B2026.02 | 62.8 | 67.9 | 68.6 | 90.4 | 71.9 | 65.9 | 69.2 | |
| Embed-RL-2BParameters=2B2026.04 | 62.8 | 67.9 | 68.6 | 90.4 | 71.9 | 65.9 | 69.2 | |
| VLM2VecParameters=7B2026.02 | 62.6 | 57.8 | 69.9 | 81.7 | 72.2 | 57.8 | 65.8 | |
| VLM2VecModel Size/Backbone=Qwen2-VL-7B2025.11 | 62.6 | 57.8 | 69.9 | 81.7 | 72.2 | 57.8 | 65.8 | |
| VLM2VecParameters=7B2026.04 | 62.6 | 57.8 | 69.9 | 81.7 | 72.2 | 57.8 | 65.8 | |
| LLaVEParameters=2B2026.02 | 62.1 | 60.2 | 65.2 | 84.9 | 69.4 | 59.8 | 65.2 | |
| LLaVE-2B2025.11 | 62.1 | 60.2 | 65.2 | 84.9 | 69.4 | 59.8 | 65.2 | |
| LLaVE-2BParameters=2B2026.04 | 62.1 | 60.2 | 65.2 | 84.9 | 69.4 | 59.8 | 65.2 | |
| VLM2VecParameters=2B2026.02 | 59 | 49.4 | 65.4 | 73.4 | 66 | 52.6 | 60.1 | |
| VLM2VecModel Size/Backbone=Qwen2-VL-2B2025.11 | 59 | 49.4 | 65.4 | 73.4 | 66 | 52.6 | 60.1 | |
| VLM2VecParameters=2B2026.04 | 59 | 49.4 | 65.4 | 73.4 | 66 | 52.6 | 60.1 | |
| MMRetParameters=7B2026.02 | 56 | 57.4 | 69.9 | 83.6 | 68 | 59.1 | 64.1 | |
| MMRet-7B2025.11 | 56 | 57.4 | 69.9 | 83.6 | 68 | 59.1 | 64.1 | |
| MMRet-7BParameters=7B2026.04 | 56 | 57.4 | 69.9 | 83.6 | 68 | 59.1 | 64.1 | |
| UniMEParameters=4B2026.02 | 54.8 | 55.9 | 64.5 | 81.8 | 68.2 | 52.7 | 64.2 | |
| UniME-4B2025.11 | 54.8 | 55.9 | 64.5 | 81.8 | 68.2 | 52.7 | 64.2 | |
| UniME-4BParameters=4B2026.04 | 54.8 | 55.9 | 64.5 | 81.8 | 68.2 | 52.7 | 64.2 | |
| OpenCLIP2026.02 | 47.8 | 10.9 | 52.3 | 53.3 | 39.3 | 40.2 | 39.7 | |
| OpenCLIP2025.11 | 47.8 | 10.9 | 52.3 | 53.3 | 39.3 | 40.2 | 39.7 | |
| OpenCLIP2026.04 | 47.8 | 10.9 | 52.3 | 53.3 | 39.3 | 40.2 | 39.7 | |
| UniIRBackbone=CLIPsf2026.02 | 44.3 | 16.2 | 61.8 | 65.3 | 47.1 | 41.7 | 44.7 | |
| UniIRBackbone=CLIP, Fusion type=score-level fusion (SF)2025.11 | 44.3 | 16.2 | 61.8 | 65.3 | 47.1 | 41.7 | 44.7 | |
| UniIRBackbone=CLIPSF2026.04 | 44.3 | 16.2 | 61.8 | 65.3 | 47.1 | 41.7 | 44.7 | |
| CLIP2026.02 | 42.8 | 9.1 | 53 | 51.8 | 37.1 | 38.7 | 37.8 | |
| CLIP2025.11 | 42.8 | 9.1 | 53 | 51.8 | 37.1 | 38.7 | 37.8 | |
| CLIP2026.04 | 42.8 | 9.1 | 53 | 51.8 | 37.1 | 38.7 | 37.8 | |
| UniIRBackbone=BLIPff2026.02 | 42.1 | 15 | 60.1 | 62.2 | 44.7 | 40.4 | 42.8 | |
| UniIRBackbone=BLIP, Fusion type=feature-level fusion (FF)2025.11 | 42.1 | 15 | 60.1 | 62.2 | 44.7 | 40.4 | 42.8 | |
| UniIRBackbone=BLIPFF2026.04 | 42.1 | 15 | 60.1 | 62.2 | 44.7 | 40.4 | 42.8 | |
| SigLIP2026.02 | 40.3 | 8.4 | 31.6 | 59.5 | 32.3 | 38 | 34.8 | |
| SigLIP2025.11 | 40.3 | 8.4 | 31.6 | 59.5 | 32.3 | 38 | 34.8 | |
| SigLIP2026.04 | 40.3 | 8.4 | 31.6 | 59.5 | 32.3 | 38 | 34.8 | |
| Magiclens2026.02 | 38.8 | 8.3 | 35.4 | 26 | 31 | 23.7 | 27.8 | |
| Magiclens2025.11 | 38.8 | 8.3 | 35.4 | 26 | 31 | 23.7 | 27.8 | |
| Magiclens2026.04 | 38.8 | 8.3 | 35.4 | 26 | 31 | 23.7 | 27.8 | |
| BLIP22026.02 | 27 | 4.2 | 33.9 | 47 | 25.3 | 25.1 | 25.2 | |
| BLIP22025.11 | 27 | 4.2 | 33.9 | 47 | 25.3 | 25.1 | 25.2 | |
| BLIP-22026.04 | 27 | 4.2 | 33.9 | 47 | 25.3 | 25.1 | 25.2 | |
| E5-V2026.02 | 21.8 | 4.9 | 11.5 | 19 | 14.9 | 11.5 | 13.3 | |
| E5-V2025.11 | 21.8 | 4.9 | 11.5 | 19 | 14.9 | 11.5 | 13.3 |