Document Retrieval on CMDR-Bench
56.6Overall nDCG@5CMDR-EmbedQwen
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CMDR-EmbedQwenBackbone=ColQwen, #Params=2.2B2026.07 | 56.6 | 64.6 | 38.4 | 64.7 | 58.9 | |
| CMDR-EmbedQwen2026.07 | 56.6 | — | — | — | — | |
| CMDR-EmbedQwenMultitask Learning=true2026.07 | 56.1 | — | — | — | — | |
| CMDR-EmbedPaliBackbone=ColPali, #Params=2.9B2026.07 | 49.8 | 53.8 | 33.3 | 57.7 | 54.3 | |
| CMDR-EmbedPali2026.07 | 49.8 | — | — | — | — | |
| CMDR-EmbedPaliMultitask Learning=true2026.07 | 47.1 | — | — | — | — | |
| ColQwenBackbone=Qwen2-VL, #Params=2.2B, Finetuned on CMDR-Synth=true2026.07 | 40.4 | 45.8 | 34.8 | 38.7 | 42.2 | |
| ColPaliBackbone=Paligemma, #Params=2.9B, Finetuned on CMDR-Synth=true2026.07 | 36.3 | 41 | 27.5 | 36.8 | 39.9 | |
| Qwen3-VL EmbeddingBackbone=Qwen3-VL, #Params=8B2026.07 | 34.4 | 38.2 | 26.6 | 35.7 | 37.2 | |
| ColQwenBackbone=Qwen2-VL, #Params=2.2B, Finetuned on CMDR-Synth=false2026.07 | 33.2 | 38 | 28.9 | 30 | 35.9 | |
| GMEBackbone=Qwen2-VL, #Params=2.2B2026.07 | 33 | 37.1 | 26 | 30.8 | 38 | |
| ColQwenTraining=finetuned on ViDoRe2026.07 | 32.6 | — | — | — | — | |
| ColPaliTraining=finetuned on ViDoRe2026.07 | 32.4 | — | — | — | — | |
| ColPaliBackbone=Paligemma, #Params=2.9B, Finetuned on CMDR-Synth=false2026.07 | 32.2 | 39.1 | 24.2 | 30.2 | 35.4 | |
| E5-VBackbone=LLaVA-1.6, #Params=8.4B2026.07 | 31 | 35.4 | 22.8 | 31.2 | 34.8 | |
| UniSEBackbone=Qwen2-VL, #Params=2.2B2026.07 | 30.2 | 32.3 | 24.6 | 30.9 | 33 | |
| BGEBackbone=BERT-base, #Params=109M2026.07 | 29.8 | 42.8 | 18.4 | 28.5 | 29.5 | |
| VisRAG-RetBackbone=MiniCPM-V, #Params=3.4B2026.07 | 29.2 | 35.3 | 19.8 | 27.8 | 33.9 | |
| DSEBackbone=Phi-3-V, #Params=4.2B2026.07 | 29.1 | 33.7 | 23 | 29.1 | 30.4 | |
| NV-Embed-v2Backbone=Mistral-7B, #Params=7.9B2026.07 | 28 | 36.2 | 16.9 | 28.6 | 30.4 | |
| E5Backbone=BERT-large, #Params=340M2026.07 | 27.2 | 36.4 | 19 | 26.5 | 27.1 | |
| ColModernVBERTBackbone=ModernBERT, #Params=250M2026.07 | 26.6 | 31.3 | 22.7 | 25 | 27.5 | |
| ContrieverBackbone=BERT-base, #Params=109M2026.07 | 25.1 | 35.7 | 15 | 23.5 | 26.4 | |
| VLM2VecBackbone=Phi-3.5-V, #Params=4.2B2026.07 | 24.7 | 27.9 | 16.4 | 26.4 | 28.2 | |
| BM252026.07 | 23.6 | 29.1 | 19.5 | 20.9 | 24.9 | |
| SigLIPBackbone=SOViT-400m, #Params=878M2026.07 | 17.8 | 24.5 | 8.7 | 15.5 | 22.2 | |
| CLIPBackbone=CLIP-large, #Params=428M2026.07 | 10.7 | 13.3 | 5.8 | 10.6 | 13.2 |