# Multimodal Retrieval on MMEB Total v2 Source: [Multimodal Retrieval on MMEB Total v2]() Collection index: [llms.txt]() Data updated: 2026-03-04T12:15:08.579Z Evaluation of model performance on the MMEB Total v2 benchmark suite, specifically focusing on the Multimodal Retrieval task. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Retrieval | | Dataset (including any stated split) | MMEB Total v2 | | Metric | Overall Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Overall Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Overall Score | Result date | Report paper | | --- | --- | --- | --- | | UME-R1 | 68.1 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | UME-R1 | 64.5 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | UME-R1 | 64.4 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | CAFe | 60.6 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | UME-R1 | 60.1 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | VLM2Vec-V2 | 58 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | GME | 57.8 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | DUME | 55.9 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | GME | 54.1 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | DUME | 52.7 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | VLM2Vec | 52.3 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | LamRA | 47.4 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | VLM2Vec | 47 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | ColPali-V1.3 | 44.4 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | LamRA | 40.4 | Not recorded | 6909887c3ef7ccd74e5e5b46 | ### UME-R1 Overall Score: 68.1. Result ID: 69a822386dbb7075b94e34c4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B","Oracle Evaluation Mode":"true"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 64.5. Result ID: 69a822386dbb7075b94e34c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 64.4. Result ID: 69a822386dbb7075b94e34c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B","Oracle Evaluation Mode":"true"}. This record does not assert independent reproduction. ### CAFe Overall Score: 60.6. Result ID: 69a822386dbb7075b94e34be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-OV","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 60.1. Result ID: 69a822386dbb7075b94e34c1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### VLM2Vec-V2 Overall Score: 58. Result ID: 69a822386dbb7075b94e34bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### GME Overall Score: 57.8. Result ID: 69a822386dbb7075b94e34b8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### DUME Overall Score: 55.9. Result ID: 69a822386dbb7075b94e34c0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### GME Overall Score: 54.1. Result ID: 69a822386dbb7075b94e34b7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### DUME Overall Score: 52.7. Result ID: 69a822386dbb7075b94e34bf. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### VLM2Vec Overall Score: 52.3. Result ID: 69a822386dbb7075b94e34bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### LamRA Overall Score: 47.4. Result ID: 69a822386dbb7075b94e34ba. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### VLM2Vec Overall Score: 47. Result ID: 69a822386dbb7075b94e34bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### ColPali-V1.3 Overall Score: 44.4. Result ID: 69a822386dbb7075b94e34b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"PaliGemma","Parameter Scale":"3B"}. This record does not assert independent reproduction. ### LamRA Overall Score: 40.4. Result ID: 69a822386dbb7075b94e34b9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.