Text-to-Video Retrieval on MSRVTT 1k (test)
87.4Recall@10DRL*
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DRL*Backbone=ViT-B/16, FLOPs (k=1000)=220.4k, DSL/Q-Norm trick=true2024.01 | 87.4 | 56.2 | 79.9 | — | 74.5 | |
| EERCF*Backbone=ViT-B/16, FLOPs (k=1000)=0.8k, DSL/Q-Norm trick=true2024.01 | 85.7 | 55 | 77.8 | — | 72.8 | |
| DRLBackbone=ViT-B/16, FLOPs (k=1000)=220.4k2024.01 | 85.4 | 48.9 | 76.3 | — | 70.2 | |
| X-CLIPBackbone=ViT-B/16, FLOPs (k=1000)=220.9k, reproduced=true2024.01 | 84.5 | 48.9 | 76.8 | — | 70.1 | |
| X2-VLM (base)# Params=255M, mode=fine-tuning2022.11 | 84.2 | 47.6 | 74.1 | — | — | |
| X2-VLM (large)# Params=593M, mode=fine-tuning2022.11 | 84.2 | 49.6 | 76.7 | — | — | |
| EERCFBackbone=ViT-B/16, FLOPs (k=1000)=0.8k2024.01 | 84.2 | 47.8 | 75.3 | — | 69.1 | |
| X-PoolBackbone=ViT-B/32, FLOPs (k=1000)=275.0k2024.01 | 84 | 44.4 | 73.3 | — | 67.2 | |
| EERCFBackbone=ViT-B/32, FLOPs (k=1000)=16.0k2024.01 | 83.9 | 44.7 | 74.2 | — | 67.6 | |
| OmniVL# Params=288M, mode=fine-tuning2022.11 | 83.8 | 47.8 | 74.2 | — | — | |
| DRLBackbone=ViT-B/32, FLOPs (k=1000)=220.4k2024.01 | 83.3 | 45.3 | 73.9 | — | 67.5 | |
| TS2-NetBackbone=ViT-B/32, FLOPs (k=1000)=6.1k, reproduced=true2024.01 | 82.7 | 43.6 | 71.1 | — | 65.8 | |
| CLIP4ClipBackbone=ViT-B/16, FLOPs (k=1000)=0.5k, reproduced=true2024.01 | 82.4 | 45.4 | 73.4 | — | 67.1 | |
| CenterCLIPBackbone=ViT-B/32, FLOPs (k=1000)=1.5k2024.01 | 81.7 | 42.9 | 71.4 | — | 65.3 | |
| X-CLIPBackbone=ViT-B/32, FLOPs (k=1000)=220.9k, reproduced=true2024.01 | 81.2 | 45.9 | 72.8 | — | 66.7 | |
| CLIP4ClipBackbone=ViT-B/32, FLOPs (k=1000)=0.5k, reproduced=true2024.01 | 80.5 | 41.4 | 70.6 | — | 64.2 | |
| CLIP-VIPBackbone=ViT-B/32, FLOPs (k=1000)=0.5k2024.01 | 79.3 | 40.6 | 70.4 | — | 63.4 | |
| All-in-one# Params=110M, mode=fine-tuning2022.11 | 77.1 | 37.9 | 68.1 | — | — | |
| mPLUGEvaluation Protocol=Zero-Shot, # Pretrain data=14M, Finetuned on COCO=true2022.05 | 75.4 | 44.3 | 66.4 | — | — | |
| TeachTextBackbone=ViT-B/32, FLOPs (k=1000)=0.8k2024.01 | 75 | 32.1 | 62.7 | — | 56.6 | |
| BLIPEvaluation Protocol=zero-shot2022.01 | 74.7 | 43.3 | 65.6 | 2 | — | |
| BLIPEvaluation Protocol=Zero-Shot, # Pretrain data=129M, Finetuned on COCO=true2022.05 | 74.7 | 43.3 | 65.6 | — | — | |
| VIOLETEvaluation Protocol=Fine-Tuning, # Pretrain data=Y180M, C3M, W2M2022.05 | 73.4 | 34.5 | 63 | — | — | |
| ALPROEvaluation Protocol=Finetuning, PT datasets=Web2M + CC3M (5.5M)2021.12 | 73.2 | 33.9 | 60.7 | 3 | — | |
| ALPROEvaluation Protocol=Fine-Tuning, # Pretrain data=C3M, W2M2022.05 | 73.2 | 33.9 | 60.7 | — | — | |
| FlorenceEvaluation Protocol=Zero-Shot, # Pretrain data=FLD900M2022.05 | 72.6 | 37.6 | 63.8 | — | — | |
| FiTEvaluation Protocol=Finetuning, PT datasets=Web2M + CC3M (5.5M)2021.12 | 70.5 | 31 | 59.5 | 3 | — | |
| SupportSetEvaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 69.3 | 30.1 | 58.5 | 3 | — | |
| mPLUGEvaluation Protocol=Zero-Shot, # Pretrain data=14M2022.05 | 68.2 | 38.1 | 59.2 | — | — | |
| VideoClipEvaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 66.8 | 30.9 | 55.4 | — | — | |
| VideoCLIPEvaluation Protocol=finetuning2022.01 | 66.8 | 30.9 | 55.4 | — | — | |
| VideoCLIPEvaluation Protocol=Fine-Tuning, # Pretrain data=How100M2022.05 | 66.8 | 30.9 | 55.4 | — | — | |
| AVLNetEvaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 66.6 | 27.1 | 55.6 | 4 | — | |
| CLIPEvaluation Protocol=Zero-Shot, # Pretrain data=WIT400M2022.05 | 60.7 | 26 | 49.4 | — | — | |
| ClipBERTEvaluation Protocol=Finetuning, PT datasets=COCO + VG (5.6M)2021.12 | 59.9 | 22 | 46.8 | 6 | — | |
| ClipBERTEvaluation Protocol=finetuning2022.01 | 59.9 | 22 | 46.8 | 6 | — | |
| VIOLETEvaluation Protocol=Zero-Shot, # Pretrain data=Y180M, W2M, C3M2022.05 | 59.7 | 25.9 | 49.5 | — | — | |
| HEROEvaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 57.7 | 16.8 | 43.4 | — | — | |
| ActBERTEvaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 56.9 | 16.3 | 42.8 | 10 | — | |
| PTP-VILTZero-shot=true2022.12 | 56.3 | 27.9 | 52.5 | 7 | — | |
| OA-TransZero-shot=true2022.12 | 55.6 | 23.4 | 47.5 | 8 | — | |
| ALPROEvaluation Protocol=Zero-shot, PT datasets=Web2M + CC3M (5.5M)2021.12 | 55.4 | 24.1 | 44.7 | 8 | — | |
| ALPROEvaluation Protocol=Zero-Shot, # Pretrain data=W2M, C3M2022.05 | 55.4 | 24.1 | 44.7 | — | — | |
| NoiseEst.Evaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 53.6 | 17.4 | 41.6 | 8 | — | |
| HT100MEvaluation Protocol=Finetuning, PT datasets=HT (100M)2021.12 | 52.8 | 14.9 | 40.2 | 9 | — | |
| FiTEvaluation Protocol=Zero-shot, PT datasets=Web2M + CC3M (5.5M)2021.12 | 51.6 | 18.7 | 39.5 | 10 | — | |
| FiTEvaluation Protocol=zero-shot2022.01 | 51.6 | 18.7 | 39.5 | 10 | — | |
| Frozen-in-timeZero-shot=true2022.12 | 51.6 | 18.7 | 39.5 | 10 | — | |
| JSFusionEvaluation Protocol=Finetuning, PT datasets=None2021.12 | 43.2 | 10.2 | 31.2 | 13 | — | |
| ActBERTEvaluation Protocol=Zero-shot, PT datasets=HT (100M)2021.12 | 33.1 | 8.6 | 23.4 | 36 | — | |
| ActBERTEvaluation Protocol=zero-shot2022.01 | 33.1 | 8.6 | 23.4 | 36 | — | |
| ActBERTZero-shot=true2022.12 | 33.1 | 8.6 | 23.4 | 36 | — | |
| MIL-NCEEvaluation Protocol=Zero-shot, PT datasets=HT (100M)2021.12 | 32.4 | 9.9 | 24 | 29.5 | — | |
| MIL-NCEEvaluation Protocol=zero-shot2022.01 | 32.4 | 9.9 | 24 | 29.5 | — | |
| MIL-NCEZero-shot=true2022.12 | 32.4 | 9.9 | 24 | 29.5 | — | |
| MIL-NCEEvaluation Protocol=Zero-Shot, # Pretrain data=How100M2022.05 | 32.4 | 9.9 | 24 | — | — | |
| SupportSetEvaluation Protocol=Zero-shot, PT datasets=HT (100M)2021.12 | 31.1 | 8.7 | 23 | 31 | — | |
| SupportSetEvaluation Protocol=zero-shot2022.01 | 31.1 | 8.7 | 23 | 31 | — | |
| VideoCLIPEvaluation Protocol=Zero-shot, PT datasets=HT (100M)2021.12 | 30 | 10.4 | 22.2 | — | — | |
| VideoCLIPEvaluation Protocol=zero-shot2022.01 | 30 | 10.4 | 22.2 | — | — | |
| VideoCLIPEvaluation Protocol=Zero-Shot, # Pretrain data=How100M2022.05 | 30 | 10.4 | 22.2 | — | — | |
| VATTEvaluation Protocol=Zero-Shot, # Pretrain data=How100M, AudSet2022.05 | 29.7 | — | — | — | — | |
| HT100MEvaluation Protocol=Zero-shot, PT datasets=HT (100M)2021.12 | 29.6 | 7.5 | 21.2 | 38 | — |