Multi-modal Retrieval on M-BEIR Global Pool 1.0 (test)
48.8VisualNews R@5 (qt->ci)U-MARVEL+
Evaluation Results
| Method | Links | |||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| U-MARVEL+Pipeline Stage=+Reranker2025.07 | 48.8 | 70.1 | 33.8 | — | 80.8 | 88.3 | — | — | — | 34.8 | 58.7 | 56.9 | — | — | — | — | 61.8 | 98.3 | 49.8 | 86 | 36.8 | 37.4 | 58.4 | 74.9 | 73.8 | |
| U-MARVELPipeline Stage=Single model2025.07 | 47.2 | 72.8 | 33.3 | — | 78.7 | 87.7 | — | — | — | 34 | 58.3 | 52.2 | — | — | — | — | 60.7 | 96.7 | 47.2 | 93.5 | 34.9 | 36 | 56 | 73.1 | 69.2 | |
| LamRAArchitecture=LMM, Supervision=Supervised, Candidate Pool=Global Pool2024.12 | 46.9 | 78 | 32.5 | 96.5 | 74.4 | 87.1 | 47.6 | 92.4 | 36.6 | 34.2 | 54 | 58.7 | 37.4 | 59.7 | 72.6 | 74 | 61.4 | — | — | — | — | — | — | — | — | |
| LamRAPipeline Stage=+Reranker2025.07 | 46.9 | 78 | 32.5 | — | 74.4 | 87.1 | — | — | — | 34.2 | 54 | 58.7 | — | — | — | — | 61.4 | 96.5 | 47.6 | 92.4 | 36.6 | 37.4 | 59.7 | 72.6 | 74 | |
| UniIR-CLIPSFArchitecture=Dual Encoder, Supervision=Supervised, Candidate Pool=Global Pool2024.12 | 42.6 | 77.9 | 17.8 | 84.7 | 59.4 | 78.8 | 42.8 | 92.3 | 17.9 | 32 | 39.2 | 24 | 24.3 | 43.9 | 60.2 | 44.6 | 48.9 | — | — | — | — | — | — | — | — | |
| UniIR-CLIPPipeline Stage=Single model2025.07 | 42.6 | 77.9 | 17.8 | — | 59.4 | 78.8 | — | — | — | 32 | 39.2 | 24 | — | — | — | — | 48.9 | 84.7 | 42.8 | 92.3 | 17.9 | 24.3 | 43.9 | 60.2 | 44.6 | |
| LamRA-RetArchitecture=LMM, Supervision=Supervised, Candidate Pool=Global Pool2024.12 | 41.3 | 75.4 | 28.7 | 85.8 | 62.5 | 81 | 39.3 | 90.4 | 30.4 | 32.1 | 48.4 | 48.7 | 33.1 | 50.5 | 70 | 60 | 54.9 | — | — | — | — | — | — | — | — | |
| LamRA-RetPipeline Stage=Single model2025.07 | 41.3 | 75.4 | 28.7 | — | 62.5 | 81 | — | — | — | 32.1 | 48.4 | 48.7 | — | — | — | — | 54.9 | 85.8 | 39.3 | 90.4 | 30.4 | 33.1 | 50.5 | 70 | 60 | |
| MM-EmbedPipeline Stage=Single model2025.07 | 41 | 71.3 | 17.1 | — | 68.8 | 85 | — | — | — | 32.4 | 42.1 | 42.3 | — | — | — | — | 52.7 | 95.9 | 41.3 | 90.1 | 18.4 | 25.7 | 50 | 64.1 | 57.7 | |
| UniIR-BLIPFFArchitecture=Dual Encoder, Supervision=Supervised, Candidate Pool=Global Pool2024.12 | 23 | 75.6 | 25.4 | 79.5 | 50.3 | 79.7 | 21.1 | 88.8 | 27.6 | 33 | 38.7 | 19.7 | 28.5 | 51.4 | 57.8 | 27.7 | 45.5 | — | — | — | — | — | — | — | — | |
| UniIR-BLIPPipeline Stage=Single model2025.07 | 23 | 75.6 | 25.4 | — | 50.3 | 79.7 | — | — | — | 33 | 38.7 | 19.7 | — | — | — | — | 45.5 | 79.5 | 21.1 | 88.8 | 27.6 | 28.5 | 51.4 | 57.8 | 27.7 |