Image-Text Retrieval on MSCOCO (IR@1, TR@1)
81.9IR@1MDPD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MDPDBackbone=BERT+BUTD, Params. (M)=5.9, Mem. (G)=3.4, QPS=24.052026.04 | 81.9 | 61.1 | — | 516.2 | |
| UniPTBackbone=BERT+BUTD, Params. (M)=5.9, Mem. (G)=3.1, QPS=12.752026.04 | 80.2 | 59.8 | — | 510.5 | |
| SSFBackbone=BERT+BUTD, Params. (M)=0.2, Mem. (G)=8.4, QPS=22.232026.04 | 80 | 60.4 | — | 512.8 | |
| AdaLoRABackbone=BERT+BUTD, Params. (M)=1.0, Mem. (G)=8.8, QPS=23.032026.04 | 79.8 | 60.1 | — | 510.3 | |
| Fully-FTBackbone=BERT+BUTD, Params. (M)=109.5, Mem. (G)=9.9, QPS=24.142026.04 | 79.7 | 62.1 | — | 513.5 | |
| FacTBackbone=BERT+BUTD, Params. (M)=0.6, Mem. (G)=8.72026.04 | 79.2 | 59.3 | — | 508.8 | |
| AdapterBackbone=BERT+BUTD, Params. (M)=2.6, Mem. (G)=8.8, QPS=22.442026.04 | 79.1 | 60.5 | — | 511.3 | |
| LoRABackbone=BERT+BUTD, Params. (M)=1.1, Mem. (G)=8.8, QPS=23.372026.04 | 78.8 | 59.6 | — | 508.2 | |
| PromptBackbone=BERT+BUTD, Params. (M)=10.7, Mem. (G)=9.4, QPS=17.642026.04 | 78.7 | 59 | — | 508.5 | |
| LSTBackbone=BERT+BUTD, Params. (M)=7.5, Mem. (G)=4.6, QPS=10.732026.04 | 77.9 | 57.3 | — | 501.9 | |
| BitFitBackbone=BERT+BUTD, Params. (M)=0.9, Mem. (G)=8.6, QPS=23.702026.04 | 77.3 | 57.8 | — | 503.9 | |
| PartiallyBackbone=BERT+BUTD, Params. (M)=0.8, Mem. (G)=1.0, QPS=24.192026.04 | 74.8 | 57.3 | — | 485.5 | |
| Cluster-aware UpcyclingArch.=MoE, Backbone=ViT-B/16, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 35.4 | 51.6 | 43.5 | — | |
| Sparse UpcyclingArch.=MoE, Backbone=ViT-B/16, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 34.9 | 50.9 | 42.9 | — | |
| DenseArch.=Dense, Backbone=ViT-B/16, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 34.3 | 50.8 | 42.6 | — | |
| Drop-UpcyclingArch.=MoE, Backbone=ViT-B/16, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 34.1 | 51.3 | 42.7 | — | |
| CLIP-MoEArch.=MoE, Backbone=ViT-B/16, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 34 | 51.5 | 42.8 | — | |
| DenseArch.=Dense, Backbone=ViT-B/16, Samples=4.0B, Zero-shot protocol=true2026.04 | 32.5 | 49.1 | 40.8 | — | |
| DeRS-LMArch.=MoE, Backbone=ViT-B/32, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 31 | 47.7 | 39.4 | — | |
| Cluster-aware UpcyclingArch.=MoE, Backbone=ViT-B/32, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 31 | 48.2 | 39.6 | — | |
| DenseArch.=Dense, Backbone=ViT-B/32, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 30.8 | 47.5 | 39.2 | — | |
| Sparse UpcyclingArch.=MoE, Backbone=ViT-B/32, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 30.8 | 48 | 39.4 | — | |
| Drop-UpcyclingArch.=MoE, Backbone=ViT-B/32, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 29.7 | 46.5 | 38.1 | — | |
| CLIP-MoEArch.=MoE, Backbone=ViT-B/32, Samples=4.0B+1.3B, Zero-shot protocol=true2026.04 | 29.5 | 46.8 | 38.2 | — | |
| DenseArch.=Dense, Backbone=ViT-B/32, Samples=4.0B, Zero-shot protocol=true2026.04 | 25.5 | 42.5 | 34 | — | |
| CLIPBackbone=RN50, Evaluation Protocol=zero-shot, Pre-training Dataset=CC3M2026.03 | 21.3 | 26.9 | 24.1 | — | |
| CLIP + MM-TSBackbone=RN50, Evaluation Protocol=zero-shot, Pre-training Dataset=CC3M2026.03 | 21.2 | 28.4 | 24.8 | — |