# Multimodal Retrieval on MMEB Image V2 Source: [Multimodal Retrieval on MMEB Image V2]() Collection index: [llms.txt]() Data updated: 2026-06-11T08:32:16.491Z Evaluation of model performance on the image-specific tasks within the MMEB V2 benchmark suite, focusing on Multimodal Retrieval capabilities measured by Classification (CLS), Question Answering (QA), Retrieval (RET), and Grounding (GD) scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Retrieval | | Dataset (including any stated split) | MMEB Image V2 | | Metric | Overall Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Overall Score](): Higher is better; unit: PERCENT. - [CLS Accuracy](): Higher is better; unit: PERCENT. - [QA Accuracy](): Higher is better; unit: PERCENT. - [Retrieval Score (RET)](): Higher is better; unit: PERCENT. - [Grounding Score (GD)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Overall Score | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-VL-Reranker-8B | 78.2 | Not recorded | 6a28f37890966ade919c19ba | | Qwen3-VL-8B-reranker | 76.2 | Not recorded | 6a28f37890966ade919c19ba | | Qwen3-VL-4B-reranker | 75.4 | Not recorded | 6a28f37890966ade919c19ba | | Qwen3-VL-8B-miniReranker | 74.3 | Not recorded | 6a28f37890966ade919c19ba | | UME-R1 | 74.2 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | Qwen3-VL-Reranker-2B | 74 | Not recorded | 6a28f37890966ade919c19ba | | Qwen3-VL-4B-miniReranker | 72.5 | Not recorded | 6a28f37890966ade919c19ba | | Qwen3-VL-2B-reranker | 71.6 | Not recorded | 6a28f37890966ade919c19ba | | UME-R1 | 71.3 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | UME-R1 | 70.2 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | Qwen3-VL-2B-miniReranker | 68.2 | Not recorded | 6a28f37890966ade919c19ba | | CAFe | 67.6 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | UME-R1 | 66.6 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | UME-R1 | 66.6 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | DUME | 66.4 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | PLUME | 66.3 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | VLM2Vec | 65.5 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | VLM2Vec-Qwen2VL (7B) | 65.5 | Not recorded | 6a28f37890966ade919c19ba | | VLM2Vec-V2 | 64.9 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | VLM2Vec-V2 | 64.9 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | DUME | 62.5 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | DUME | 62.5 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | Qwen3-VL-2B-Instruct | 61.7 | Not recorded | 6a28f37890966ade919c19ba | | VLM2Vec | 59.7 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | VLM2Vec | 59.7 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | VLM2Vec (2B) | 59.7 | Not recorded | 6a28f37890966ade919c19ba | | GME | 56 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | GME (7B) | 56 | Not recorded | 6a28f37890966ade919c19ba | | LamRA | 54.1 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | LamRA | 54.1 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | LamRA | 52.4 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | LamRA-Qwen2.5 (7B) | 52.4 | Not recorded | 6a28f37890966ade919c19ba | | GME | 51.9 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | GME | 51.9 | Not recorded | 69cf49bb0636e5bf4afcdba9 | | GME (2B) | 51.9 | Not recorded | 6a28f37890966ade919c19ba | | ColPali-V1.3 | 34.9 | Not recorded | 6909887c3ef7ccd74e5e5b46 | | ColPali v1.3 (3B) | 34.9 | Not recorded | 6a28f37890966ade919c19ba | ### Qwen3-VL-Reranker-8B Overall Score: 78.2. Result ID: 6a2a7275fbc373592c278fdd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"8B"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-reranker Overall Score: 76.2. Result ID: 6a2a7275fbc373592c278fde. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"8B","Fine-tuning Recipe":"our training recipe"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-reranker Overall Score: 75.4. Result ID: 6a2a7275fbc373592c278fd8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"4B","Fine-tuning Recipe":"our training recipe"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-miniReranker Overall Score: 74.3. Result ID: 6a2a7275fbc373592c278fdf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"8B","Fine-tuning Recipe":"our training recipe","Reranker Type":"miniReranker"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 74.2. Result ID: 69a822386dbb7075b94e3497. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B","Oracle Evaluation Mode":"true"}. This record does not assert independent reproduction. ### Qwen3-VL-Reranker-2B Overall Score: 74. Result ID: 6a2a7275fbc373592c278fd4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"2B"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-miniReranker Overall Score: 72.5. Result ID: 6a2a7275fbc373592c278fd9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"4B","Fine-tuning Recipe":"our training recipe","Reranker Type":"miniReranker"}. This record does not assert independent reproduction. ### Qwen3-VL-2B-reranker Overall Score: 71.6. Result ID: 6a2a7275fbc373592c278fd6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"2B","Fine-tuning Recipe":"our training recipe"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 71.3. Result ID: 69a822386dbb7075b94e3495. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 70.2. Result ID: 69a822386dbb7075b94e3496. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B","Oracle Evaluation Mode":"true"}. This record does not assert independent reproduction. ### Qwen3-VL-2B-miniReranker Overall Score: 68.2. Result ID: 6a2a7275fbc373592c278fd7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"2B","Fine-tuning Recipe":"our training recipe","Reranker Type":"miniReranker"}. This record does not assert independent reproduction. ### CAFe Overall Score: 67.6. Result ID: 69a822386dbb7075b94e3491. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaVA-OV","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 66.6. Result ID: 69a822386dbb7075b94e3494. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### UME-R1 Overall Score: 66.6. Result ID: 69cf830447f34ce924b9976d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Venue":"ICLR’26","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### DUME Overall Score: 66.4. Result ID: 69a822386dbb7075b94e3493. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### PLUME Overall Score: 66.3. Result ID: 69cf830447f34ce924b9976e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Venue":"Ours","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### VLM2Vec Overall Score: 65.5. Result ID: 69a822386dbb7075b94e348f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### VLM2Vec-Qwen2VL (7B) Overall Score: 65.5. Result ID: 6a2a7275fbc373592c278fdc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B"}. This record does not assert independent reproduction. ### VLM2Vec-V2 Overall Score: 64.9. Result ID: 69a822386dbb7075b94e3490. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### VLM2Vec-V2 Overall Score: 64.9. Result ID: 69cf830447f34ce924b9976b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Venue":"TMLR’26","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### DUME Overall Score: 62.5. Result ID: 69a822386dbb7075b94e3492. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### DUME Overall Score: 62.5. Result ID: 69cf830447f34ce924b9976c. [Reporting paper]() Recorded evaluation configuration: {"Venue":"ICLR’26","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### Qwen3-VL-2B-Instruct Overall Score: 61.7. Result ID: 6a2a7275fbc373592c278fd5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"2B"}. This record does not assert independent reproduction. ### VLM2Vec Overall Score: 59.7. Result ID: 69a822386dbb7075b94e348e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### VLM2Vec Overall Score: 59.7. Result ID: 69cf830447f34ce924b99769. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Venue":"ICLR’25","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### VLM2Vec (2B) Overall Score: 59.7. Result ID: 6a2a7275fbc373592c278fd3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"2B"}. This record does not assert independent reproduction. ### GME Overall Score: 56. Result ID: 69a822386dbb7075b94e348b. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### GME (7B) Overall Score: 56. Result ID: 6a2a7275fbc373592c278fda. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B"}. This record does not assert independent reproduction. ### LamRA Overall Score: 54.1. Result ID: 69a822386dbb7075b94e348c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### LamRA Overall Score: 54.1. Result ID: 69cf830447f34ce924b99768. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Venue":"CVPR’25","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### LamRA Overall Score: 52.4. Result ID: 69a822386dbb7075b94e348d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-VL","Parameter Scale":"7B"}. This record does not assert independent reproduction. ### LamRA-Qwen2.5 (7B) Overall Score: 52.4. Result ID: 6a2a7275fbc373592c278fdb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"7B"}. This record does not assert independent reproduction. ### GME Overall Score: 51.9. Result ID: 69a822386dbb7075b94e348a. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-VL","Parameter Scale":"2B"}. This record does not assert independent reproduction. ### GME Overall Score: 51.9. Result ID: 69cf830447f34ce924b9976a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Venue":"CVPR’25","Backbone":"Qwen2-VL-2B"}. This record does not assert independent reproduction. ### GME (2B) Overall Score: 51.9. Result ID: 6a2a7275fbc373592c278fd2. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"2B"}. This record does not assert independent reproduction. ### ColPali-V1.3 Overall Score: 34.9. Result ID: 69a822386dbb7075b94e3489. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"PaliGemma","Parameter Scale":"3B"}. This record does not assert independent reproduction. ### ColPali v1.3 (3B) Overall Score: 34.9. Result ID: 6a2a7275fbc373592c278fd1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"3B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.