# Multi-modal Representation Learning on MMEB In-Distribution 1.0 Source: [Multi-modal Representation Learning on MMEB In-Distribution 1.0]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:50:00.671Z Evaluation of model performance on the In-Distribution 1.0 version of the MMEB benchmark suite, focusing on multi-modal representation learning capabilities, measured via Precision@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-modal Representation Learning | | Dataset (including any stated split) | MMEB In-Distribution 1.0 | | Metric | MMEB IND Precision@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MMEB IND Precision@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | MMEB IND Precision@1 | Result date | Report paper | | --- | --- | --- | --- | | SDE | 71.6 | Not recorded | 698c0cecc97a0b3e3b5e103f | | SDE | 69.8 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniME | 68.4 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniME | 68.2 | Not recorded | 698c0cecc97a0b3e3b5e103f | | MegaPairs | 68 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 67.5 | Not recorded | 698c0cecc97a0b3e3b5e103f | | SDE | 66.9 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 66.5 | Not recorded | 698c0cecc97a0b3e3b5e103f | | SDE | 66.3 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 66 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 61 | Not recorded | 698c0cecc97a0b3e3b5e103f | | OpenCLIP-FFT | 50.5 | Not recorded | 698c0cecc97a0b3e3b5e103f | | CLIP-FFT | 47.6 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniIR | 47.1 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniIR | 44.7 | Not recorded | 698c0cecc97a0b3e3b5e103f | | CLIP | 37.1 | Not recorded | 698c0cecc97a0b3e3b5e103f | | Magiclens | 31 | Not recorded | 698c0cecc97a0b3e3b5e103f | | BLIP2 | 25.3 | Not recorded | 698c0cecc97a0b3e3b5e103f | ### SDE MMEB IND Precision@1: 71.6. Result ID: 6998daf95698665ec43f017e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"1344x1344 (HR)"}. This record does not assert independent reproduction. ### SDE MMEB IND Precision@1: 69.8. Result ID: 6998daf95698665ec43f0178. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Phi3.5-V"}. This record does not assert independent reproduction. ### UniME MMEB IND Precision@1: 68.4. Result ID: 6998daf95698665ec43f0175. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6"}. This record does not assert independent reproduction. ### UniME MMEB IND Precision@1: 68.2. Result ID: 6998daf95698665ec43f0174. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Phi3.5-V"}. This record does not assert independent reproduction. ### MegaPairs MMEB IND Precision@1: 68. Result ID: 6998daf95698665ec43f0176. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6"}. This record does not assert independent reproduction. ### VLM2Vec MMEB IND Precision@1: 67.5. Result ID: 6998daf95698665ec43f017d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"1344x1344 (HR)"}. This record does not assert independent reproduction. ### SDE MMEB IND Precision@1: 66.9. Result ID: 6998daf95698665ec43f017a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### VLM2Vec MMEB IND Precision@1: 66.5. Result ID: 6998daf95698665ec43f0177. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Phi3.5-V"}. This record does not assert independent reproduction. ### SDE MMEB IND Precision@1: 66.3. Result ID: 6998daf95698665ec43f017c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"336x336 (LR)"}. This record does not assert independent reproduction. ### VLM2Vec MMEB IND Precision@1: 66. Result ID: 6998daf95698665ec43f0179. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### VLM2Vec MMEB IND Precision@1: 61. Result ID: 6998daf95698665ec43f017b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"336x336 (LR)"}. This record does not assert independent reproduction. ### OpenCLIP-FFT MMEB IND Precision@1: 50.5. Result ID: 6998daf95698665ec43f0173. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction. ### CLIP-FFT MMEB IND Precision@1: 47.6. Result ID: 6998daf95698665ec43f0172. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction. ### UniIR MMEB IND Precision@1: 47.1. Result ID: 6998daf95698665ec43f0170. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot","Fusion Strategy":"score-level","Backbone":"CLIP"}. This record does not assert independent reproduction. ### UniIR MMEB IND Precision@1: 44.7. Result ID: 6998daf95698665ec43f016f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot","Fusion Strategy":"feature-level","Backbone":"BLIP"}. This record does not assert independent reproduction. ### CLIP MMEB IND Precision@1: 37.1. Result ID: 6998daf95698665ec43f016d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Magiclens MMEB IND Precision@1: 31. Result ID: 6998daf95698665ec43f0171. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### BLIP2 MMEB IND Precision@1: 25.3. Result ID: 6998daf95698665ec43f016e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.