# Multi-modal Representation Learning on MMEB Overall 1.0 Source: [Multi-modal Representation Learning on MMEB Overall 1.0]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:39:13.429Z Evaluation of model performance on the MMEB Overall 1.0 benchmark suite, focusing on multi-modal representation learning capabilities measured by Precision@1 scores across Classification, VQA, Retrieval, and Grounding tasks. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multi-modal Representation Learning | | Dataset (including any stated split) | MMEB Overall 1.0 | | Metric | Classification P@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Classification P@1](): Higher is better; unit: PERCENT. - [VQA P@1](): Higher is better; unit: PERCENT. - [Retrieval P@1](): Higher is better; unit: PERCENT. - [Grounding P@1](): Higher is better; unit: PERCENT. - [Overall P@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Classification P@1 | Result date | Report paper | | --- | --- | --- | --- | | SDE | 61.6 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 61.2 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniME | 60.6 | Not recorded | 698c0cecc97a0b3e3b5e103f | | SDE | 60 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 59 | Not recorded | 698c0cecc97a0b3e3b5e103f | | SDE | 57.2 | Not recorded | 698c0cecc97a0b3e3b5e103f | | OpenCLIP-FFT | 56 | Not recorded | 698c0cecc97a0b3e3b5e103f | | MegaPairs | 56 | Not recorded | 698c0cecc97a0b3e3b5e103f | | CLIP-FFT | 55.2 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 54.8 | Not recorded | 698c0cecc97a0b3e3b5e103f | | VLM2Vec | 54.7 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniME | 54.6 | Not recorded | 698c0cecc97a0b3e3b5e103f | | SDE | 54.6 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniIR | 44.3 | Not recorded | 698c0cecc97a0b3e3b5e103f | | CLIP | 42.8 | Not recorded | 698c0cecc97a0b3e3b5e103f | | UniIR | 42.1 | Not recorded | 698c0cecc97a0b3e3b5e103f | | Magiclens | 38.8 | Not recorded | 698c0cecc97a0b3e3b5e103f | | BLIP2 | 27 | Not recorded | 698c0cecc97a0b3e3b5e103f | ### SDE Classification P@1: 61.6. Result ID: 6998daf95698665ec43f016c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"1344x1344 (HR)"}. This record does not assert independent reproduction. ### VLM2Vec Classification P@1: 61.2. Result ID: 6998daf95698665ec43f016b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"1344x1344 (HR)"}. This record does not assert independent reproduction. ### UniME Classification P@1: 60.6. Result ID: 6998daf95698665ec43f0163. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6"}. This record does not assert independent reproduction. ### SDE Classification P@1: 60. Result ID: 6998daf95698665ec43f0168. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### VLM2Vec Classification P@1: 59. Result ID: 6998daf95698665ec43f0167. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### SDE Classification P@1: 57.2. Result ID: 6998daf95698665ec43f016a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"336x336 (LR)"}. This record does not assert independent reproduction. ### OpenCLIP-FFT Classification P@1: 56. Result ID: 6998daf95698665ec43f0161. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction. ### MegaPairs Classification P@1: 56. Result ID: 6998daf95698665ec43f0164. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6"}. This record does not assert independent reproduction. ### CLIP-FFT Classification P@1: 55.2. Result ID: 6998daf95698665ec43f0160. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning"}. This record does not assert independent reproduction. ### VLM2Vec Classification P@1: 54.8. Result ID: 6998daf95698665ec43f0165. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Phi3.5-V"}. This record does not assert independent reproduction. ### VLM2Vec Classification P@1: 54.7. Result ID: 6998daf95698665ec43f0169. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"LLaVA-1.6","Resolution":"336x336 (LR)"}. This record does not assert independent reproduction. ### UniME Classification P@1: 54.6. Result ID: 6998daf95698665ec43f0162. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Phi3.5-V"}. This record does not assert independent reproduction. ### SDE Classification P@1: 54.6. Result ID: 6998daf95698665ec43f0166. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Fine-tuning","Model Backbone":"Phi3.5-V"}. This record does not assert independent reproduction. ### UniIR Classification P@1: 44.3. Result ID: 6998daf95698665ec43f015e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot","Fusion Strategy":"score-level","Backbone":"CLIP"}. This record does not assert independent reproduction. ### CLIP Classification P@1: 42.8. Result ID: 6998daf95698665ec43f015b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### UniIR Classification P@1: 42.1. Result ID: 6998daf95698665ec43f015d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot","Fusion Strategy":"feature-level","Backbone":"BLIP"}. This record does not assert independent reproduction. ### Magiclens Classification P@1: 38.8. Result ID: 6998daf95698665ec43f015f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### BLIP2 Classification P@1: 27. Result ID: 6998daf95698665ec43f015c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Evaluation Protocol":"Zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.