Multimodal Retrieval on Multi30k (test)
97.8Recall (EN)CCLM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CCLMSize=large, Fine-tune setting=all-language2022.06 | 97.8 | 95.8 | 96.6 | 96.2 | — | — | |
| CCLMSize=base, Fine-tune setting=all-language2022.06 | 97.2 | 94.6 | 95.5 | 94.8 | — | — | |
| ALIGN-L2Evaluation Protocol=Fine-tuned, Data=AT-en, Type=Dual Encoder2021.09 | 96 | — | — | — | — | — | |
| CCLMPre-training data=3M, Fine-tune setting=all-language2022.06 | 95.3 | 92.4 | 92.1 | 91.2 | — | — | |
| MURAL-LARGEEvaluation Protocol=Fine-tuned, Data=AT+MBT, Type=Dual Encoder2021.09 | 93.8 | 90.4 | 89.9 | 87.1 | — | — | |
| MURALSize=large, Fine-tune setting=all-language2022.06 | 93.8 | 90.4 | 89.9 | 87.1 | — | — | |
| ALIGN-BASEEvaluation Protocol=Fine-tuned, Data=AT, Type=Dual Encoder2021.09 | 92.3 | 88.3 | 78.8 | 81.4 | — | — | |
| ALIGN_ENProtocol=zero-shot, Training Data=EN-only2021.02 | 92.2 | — | — | — | — | — | |
| ALIGN-L2Evaluation Protocol=Zero-shot, Data=AT-en, Type=Dual Encoder2021.09 | 92.2 | — | — | — | — | — | |
| ALIGN-BASEEvaluation Protocol=Fine-tuned, Data=TrTrain(AT-en), Type=Dual Encoder2021.09 | 92.2 | 88.5 | 88.1 | 84.5 | — | — | |
| MURAL-BASEEvaluation Protocol=Fine-tuned, Data=AT+MBT, Type=Dual Encoder2021.09 | 92.2 | 88.6 | 87.6 | 84.2 | — | — | |
| MURALSize=base, Fine-tune setting=all-language2022.06 | 92.2 | 88.6 | 87.6 | 84.2 | — | — | |
| MURAL-BASEEvaluation Protocol=Fine-tuned, Data=TrTrain(CC12m)+EOBT, Type=Dual Encoder2021.09 | 91 | 87.3 | 86.4 | 82.4 | — | — | |
| MURALTraining Data=CC12m + SBU2022.11 | 91 | 87.3 | 86.4 | 82.4 | 86.77 | — | |
| ALIGN_mlingProtocol=zero-shot, Training Data=multilingual2021.02 | 90.2 | 84.1 | 84.9 | 63.2 | — | — | |
| MURAL-LARGEEvaluation Protocol=Zero-shot, Data=AT+MBT, Type=Dual Encoder2021.09 | 89.2 | 83.5 | 83.1 | 77 | — | — | |
| ERNIE-UniX22022.11 | 88.8 | 84.71 | 84.05 | 82.45 | 85 | — | |
| UC2Protocol=w/ fine-tuning2021.02 | 88.2 | 84.5 | 83.9 | 81.2 | — | — | |
| UC2Evaluation Protocol=Fine-tuned, Data=TrTrain(CC3m), Type=Cross Encoder2021.09 | 88.2 | 84.5 | 83.9 | 81.2 | — | — | |
| UC22022.11 | 88.2 | 84.5 | 83.9 | 81.2 | 84.45 | — | |
| UC2Fine-tune setting=all-language2022.06 | 88.2 | 84.5 | 83.9 | 81.2 | — | — | |
| M3PProtocol=w/ fine-tuning2021.02 | 87.7 | 82.7 | 73.9 | 72.2 | — | — | |
| M3PEvaluation Protocol=Fine-tuned, Data=CC3m+Wiki, Type=Cross Encoder2021.09 | 87.7 | 82.7 | 73.9 | 72.2 | — | — | |
| M3PFine-tune setting=all-language2022.06 | 87.7 | 82.7 | 73.9 | 72.2 | — | — | |
| M3P2022.11 | 87.4 | 82.7 | 73.9 | 72.2 | 79.05 | — | |
| ALIGN-BASE-ENEvaluation Protocol=Zero-shot, Data=AT-en-translate-test, Type=Dual Encoder2021.09 | 84.3 | 78.9 | 78.3 | 71.1 | — | — | |
| CCLM-base-3MSetting=Zero-shot, Metric=Average Recall@K (K=1, 5, 10)2022.06 | 83.7 | 79.1 | 76.7 | 73.9 | — | — | |
| ALIGN-BASEEvaluation Protocol=Zero-shot, Data=AT, Type=Dual Encoder2021.09 | 83.3 | 75 | 74.2 | 47.9 | — | — | |
| MURAL-BASEEvaluation Protocol=Zero-shot, Data=AT+MBT, Type=Dual Encoder2021.09 | 82.4 | 76.2 | 75 | 64.6 | — | — | |
| MURAL-baseSetting=Zero-shot, Metric=Average Recall@K (K=1, 5, 10)2022.06 | 82.4 | 76.2 | 75 | 64.6 | — | — | |
| ALIGN-BASEEvaluation Protocol=Zero-shot, Data=TrTrain(AT-en), Type=Dual Encoder2021.09 | 82 | 75.2 | 74.7 | 68.2 | — | — | |
| MURAL-BASEEvaluation Protocol=Zero-shot, Data=TrTrain(CC12m)+EOBT, Type=Dual Encoder2021.09 | 80.9 | 76 | 75.7 | 68.2 | — | — | |
| SMALREvaluation Protocol=Fine-tuned, Data=no pretraining, Type=Dual Encoder2021.09 | 74.5 | 69.8 | 65.9 | 64.8 | — | — | |
| M3PProtocol=zero-shot2021.02 | 57.9 | 36.8 | 27.1 | 20.4 | — | — | |
| M3PEvaluation Protocol=Zero-shot, Data=CC3m+Wiki, Type=Cross Encoder2021.09 | 57.9 | 36.8 | 27.1 | 20.4 | — | — | |
| Jina-Clip-v22026.01 | — | — | — | — | — | 92.1 | |
| LaBSEBackbone=ViT-L/142026.01 | — | — | — | — | — | 87.9 | |
| Ours-stage3Training Configuration=Full three-stage model including NLU2026.01 | — | — | — | — | — | 94.8 | |
| SigLIP22026.01 | — | — | — | — | — | 87.9 | |
| XLM-R LargeBackbone=ViT-B/322026.01 | — | — | — | — | — | 88.3 | |
| XLM-R LargeBackbone=ViT-L/142026.01 | — | — | — | — | — | 90.8 | |
| XLM-R LargeBackbone=ViT-B/16+2026.01 | — | — | — | — | — | 92.9 |