Multimodal Embedding on MMEB (test)
56Precision@1 (Classification)OpenCLIP
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OpenCLIPtuning=supervised fine-tuning2024.12 | 56 | 21.9 | 55.4 | 64.1 | 47.2 | 50.5 | 43.1 | |
| MMRet-MLLMBackbone=LLaVA-1.6, tuning=supervised fine-tuning2024.12 | 56 | 57.4 | 69.9 | 83.6 | 64.1 | 68 | 59.1 | |
| CLIPtuning=supervised fine-tuning2024.12 | 55.2 | 19.7 | 53.2 | 62.2 | 45.4 | 47.6 | 42.8 | |
| VLM2VecBackbone=Phi-3.5-V, tuning=supervised fine-tuning2024.12 | 54.8 | 54.9 | 62.3 | 79.5 | 60.1 | 66.5 | 52 | |
| VLM2VecBackbone=LLaVA-1.6, tuning=supervised fine-tuning2024.12 | 54.7 | 50.3 | 56.2 | 64 | 55 | 61 | 47.5 | |
| OpenCLIPevaluation protocol=zero-shot2024.12 | 0.478 | 0.109 | 0.523 | 0.533 | 0.397 | — | — | |
| MMRet-MLLMevaluation protocol=zero-shot, backbone=LLaVA-1.6, trained_on=MegaPairs2024.12 | 0.472 | 0.184 | 0.565 | 0.622 | 0.44 | — | — | |
| CLIPevaluation protocol=zero-shot2024.12 | 0.428 | 0.091 | 0.53 | 0.518 | 0.378 | — | — | |
| UniIRevaluation protocol=zero-shot, trained_on=M-BEIR2024.12 | 0.421 | 0.15 | 0.601 | 0.622 | 0.428 | — | — | |
| SigLIPevaluation protocol=zero-shot2024.12 | 0.403 | 0.084 | 0.316 | 0.595 | 0.348 | — | — | |
| MagicLensevaluation protocol=zero-shot2024.12 | 0.388 | 0.083 | 0.354 | 0.26 | 0.278 | — | — | |
| BLIP2evaluation protocol=zero-shot2024.12 | 0.27 | 0.042 | 0.339 | 0.47 | 0.252 | — | — | |
| E5-Vevaluation protocol=zero-shot, backbone=LLaVA-1.62024.12 | 0.218 | 0.049 | 0.115 | 0.19 | 0.133 | — | — |