Text-to-Video Retrieval on MSR-VTT (1k-A)
90.6R@10HVP-Net
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| HVP-NetBackbone=CLIP ViT-B/32, Frames=12, Resolution=224x2242026.01 | 90.6 | 56.7 | 83.9 | 1 | 5.3 | — | |
| STANBackbone=CLIP-ViT-B/16, DSL post-processing=true, re-training=false2023.09 | 87.8 | 54.1 | 79.5 | 1 | — | 221.4 | |
| DRLBackbone=CLIP-ViT-B/16, DSL post-processing=false, re-training=false2023.09 | 87.6 | 53.3 | 80.3 | 1 | — | 221.2 | |
| TS2-NetBackbone=CLIP-ViT-B/16, DSL post-processing=true, re-training=true2023.09 | 87.4 | 52.8 | 79 | 1 | 11.4 | 219.2 | |
| BaselineBackbone=CLIP-ViT-B/16, DSL post-processing=true, re-training=false2023.09 | 87.1 | 53.3 | 78.8 | 1 | 11 | 219.2 | |
| TPM-CLBackbone=CLIP-ViT-B/16, DSL post-processing=true, re-training=false2023.09 | 87.1 | 55.5 | 79.4 | 1 | 10 | 222 | |
| RAPBackbone=CLIP-ViT-B/16, Trainable Params (MB)=1.06, DSL post-processing=true2024.05 | 86.7 | 52.1 | 77.3 | — | 10 | — | |
| TPM-CLBackbone=CLIP-ViT-B/32, DSL post-processing=true, re-training=false2023.09 | 86.2 | 51.7 | 77.6 | 1 | 11.4 | 215.5 | |
| OurstiBackbone=CLIP-ViT-B/162023.09 | 85.9 | 49.3 | 77 | 2 | 12.7 | 212.2 | |
| TS2-NetBackbone=CLIP-ViT-B/32, DSL post-processing=true, re-training=true2023.09 | 85.9 | 50.5 | 76.7 | 1 | 11.8 | 213.1 | |
| BaselineBackbone=CLIP-ViT-B/32, DSL post-processing=true, re-training=false2023.09 | 85.8 | 49.8 | 76.9 | 2 | 11.3 | 212.5 | |
| MUSE2026.01 | 85.6 | 50.9 | 76.7 | 1 | 10.9 | — | |
| OurswtiBackbone=CLIP-ViT-B/162023.09 | 85.5 | 49.9 | 75.8 | 2 | 12.5 | 211.2 | |
| TS2-NetBackbone=ViT16, Patch Size=16x162022.07 | 85.3 | 49.4 | 75.6 | 2 | 13.5 | — | |
| TS2-NetBackbone=CLIP-ViT-B/16, re-training=true2023.09 | 85.2 | 47.8 | 76.8 | 2 | 13.7 | 209.8 | |
| X-CLIP2026.01 | 84.8 | 49.3 | 75.8 | 2 | 12.2 | — | |
| CAMOEBackbone=CLIP-ViT-B/32, DSL post-processing=true, re-training=false2023.09 | 84.5 | 47.3 | 74.2 | 2 | 11.9 | 206 | |
| BaselineBackbone=CLIP-ViT-B/162023.09 | 84.4 | 48.6 | 74.8 | 2 | 13.6 | 207.8 | |
| OurswtiBackbone=CLIP-ViT-B/322023.09 | 84.2 | 46.9 | 74.6 | 2 | 12.8 | 205.7 | |
| DRL+Backbone=CLIP-ViT-B/16, re-training=true2023.09 | 84.2 | 49.4 | 76.4 | 2 | 13.2 | 210 | |
| OurstiBackbone=CLIP-ViT-B/322023.09 | 84.1 | 46.6 | 75 | 2 | 13.3 | 205.7 | |
| BiHSSP2026.01 | 84.1 | 48.1 | 74 | 2 | 12.1 | — | |
| TS2-Net2022.07 | 83.8 | 47 | 74.5 | 2 | 13 | — | |
| XCLIPBackbone=CLIP-ViT-B/162023.09 | 83.7 | 49 | 76.9 | 2 | 13.6 | 209.6 | |
| CLIP2TV-ViT16Type=Ours, Backbone=ViT-16, Similarity Distillation=true2021.11 | 83.6 | 49.3 | 74.7 | 2 | 13.5 | — | |
| DRLBackbone=CLIP-ViT-B/32, re-training=true2023.09 | 83.6 | 47.5 | 73.8 | 2 | 13.3 | 204.9 | |
| CLIP2TVBackbone=CLIP-ViT-B/162023.09 | 83.6 | 49.3 | 74.7 | 2 | 13.5 | 207.6 | |
| BaselineBackbone=CLIP-ViT-B/322023.09 | 83.5 | 45.3 | 74.2 | 2 | 13 | 203 | |
| STANBackbone=CLIP-ViT-B/32, DSL post-processing=true, re-training=false2023.09 | 83.5 | 49 | 74.8 | 2 | — | 207.3 | |
| XCLIPBackbone=CLIP-ViT-B/322023.09 | 83.1 | 47.4 | 73.4 | 2 | 13.7 | 203.9 | |
| TS2-NetBackbone=CLIP-ViT-B/32, re-training=true2023.09 | 83.1 | 47.2 | 73.7 | 2 | 13.1 | 204 | |
| QB-NormBackbone=CLIP-ViT-B/32, DSL post-processing=false, re-training=false2023.09 | 83 | 47.2 | 73 | 2 | — | 203.2 | |
| CLIP2TV+SDType=Ours, Backbone=ViT-32, Similarity Distillation=true2021.11 | 82.9 | 46.1 | 72.5 | 2 | 15.2 | — | |
| CLIP2TVBackbone=CLIP-ViT-B/322023.09 | 82.9 | 46.1 | 72.5 | 2 | 15.2 | 201.5 | |
| CLIP2TVPatch Size=16x162022.07 | 82.8 | 48.3 | 74.6 | 2 | 14.9 | — | |
| CLIP2022.07 | 82.2 | 39.7 | 72.3 | 2 | 12.8 | — | |
| XPoolBackbone=CLIP-ViT-B/322023.09 | 82.2 | 46.9 | 72.8 | 2 | 14.3 | 201.9 | |
| MV-AdapterBackbone=CLIP-ViT-B/16, Trainable Params (MB)=3.872024.05 | 82.1 | 46 | 72 | — | — | — | |
| CenterCLIPBackbone=CLIP-ViT-B/322023.09 | 82.1 | 44.2 | 71.6 | 2 | 15.1 | 197.9 | |
| RAPBackbone=CLIP-ViT-B/16, Trainable Params (MB)=1.062024.05 | 82 | 46.5 | 73.9 | — | 12.1 | — | |
| CenterCLIPBackbone=CLIP-ViT-B/162023.09 | 82 | 48.4 | 73.8 | 2 | 13.8 | 204.2 | |
| CAMOE2022.07 | 81.8 | 44.6 | 72.6 | 2 | 13.3 | — | |
| CLIP2Video2022.07 | 81.7 | 45.6 | 72.6 | 2 | 14.6 | — | |
| CLIP2VideoType=CLIP-based2021.11 | 81.7 | 45.6 | 72.6 | 2 | 14.6 | — | |
| CLIP2VideoTest-set=1k-A2021.06 | 81.7 | 45.6 | 72.6 | 2 | 14.6 | — | |
| CLIP4Clip2022.07 | 81.6 | 44.5 | 71.4 | 2 | 15.3 | — | |
| CLIP4Clip-seqTransfType=CLIP-based2021.11 | 81.6 | 44.5 | 71.4 | 2 | 15.3 | — | |
| CLIP4ClipBackbone=CLIP-ViT-B/322023.09 | 81.6 | 44.5 | 71.4 | 2 | 15.3 | 197.5 | |
| CLIP4Clip2026.01 | 81.6 | 44.5 | 71.4 | 2 | 15.3 | — | |
| CLIP4Clip-seqTransfTest-set=1k-A, aggregation=temporal transformer2021.06 | 81.6 | 44.5 | 71.4 | 2 | 15.3 | — | |
| RAPType=Adapter, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=1.062024.05 | 81.5 | 44.8 | 71.4 | — | 14.4 | — | |
| CLIP4Clip-meanPTrainD=W+M, E2E=false, Training Split Protocol=Training-7K2021.04 | 81.4 | 42.1 | 71.9 | 2 | 15.7 | — | |
| CLIP4ClipBackbone=CLIP-ViT-B/16, Trainable Params (MB)=149.622024.05 | 81.1 | 45.4 | 72.1 | — | 14.5 | — | |
| VoP-PBackbone=CLIP-ViT-B/16, Trainable Params (MB)=0.502024.05 | 80.9 | 43.9 | 70 | — | 12.9 | — | |
| CLIP4Clip-meanPType=CLIP-based2021.11 | 80.8 | 43.1 | 70.4 | 2 | 16.2 | — | |
| CLIP2TVType=Ours, Backbone=ViT-32, Similarity Distillation=false2021.11 | 80.8 | 45.6 | 71.1 | 2 | 15 | — | |
| CLIP4Clip-meanPTest-set=1k-A, aggregation=mean pooling2021.06 | 80.8 | 43.1 | 70.4 | 2 | 16.2 | — | |
| VoP-CBackbone=CLIP-ViT-B/16, Trainable Params (MB)=14.302024.05 | 80.2 | 44.6 | 71.8 | — | 14.6 | — | |
| ST-AdapterType=Adapter, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=77.45, Trainable text-encoder=true2024.05 | 80.1 | 42.5 | 70 | — | 17 | — | |
| CLIP4ClipType=Fine-tune, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=151.282024.05 | 79.9 | 42.6 | 70.8 | — | 16.1 | — | |
| MDMMT2022.07 | 79.7 | 38.9 | 69 | 2 | 16.5 | — | |
| MDMMTType=CLIP-based2021.11 | 79.7 | 38.9 | 69 | 2 | 16.5 | — | |
| MDMMTTest-set=1k-A2021.06 | 79.7 | 38.9 | 69 | 2 | 16.5 | — | |
| VoP-CType=Prompt, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=14.302024.05 | 79 | 40.8 | 68.1 | — | 15.8 | — | |
| LoRAType=Adapter, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=0.492024.05 | 78.9 | 40.5 | 67.1 | — | 16.4 | — | |
| CLIP4Clip-seqLSTMTrainD=W+M, E2E=false, Training Split Protocol=Training-7K2021.04 | 78.7 | 41.7 | 68.8 | 2 | 16.6 | — | |
| CLIP4Clip-seqTransfTrainD=W+M, E2E=false, Training Split Protocol=Training-7K2021.04 | 78.7 | 42 | 68.6 | 2 | 16.2 | — | |
| VPTType=Prompt, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=63.43, Trainable text-encoder=true2024.05 | 78.6 | 40.5 | 67.3 | — | 17.9 | — | |
| SSFType=Adapter, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=0.342024.05 | 78.6 | 40.8 | 68.2 | — | 17 | — | |
| CLIP4Clip-tightTransfTrainD=W+M, E2E=false, Training Split Protocol=Training-7K2021.04 | 78.4 | 37.8 | 68.4 | 2 | 17.2 | — | |
| HD-VILAPre-training set=HD-VILA-100M, Pre-training pairs=100M2024.04 | 78 | 35.6 | 65.3 | — | — | — | |
| VoP-PType=Prompt, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=0.502024.05 | 77.7 | 40.1 | 65.7 | — | 16.9 | — | |
| All-in-oneE2E=true, PT Dataset=HowTo100M, WebVid2M, # Pairs=122.5M, # Frames=92022.12 | 77.1 | 37.9 | 68.1 | — | — | — | |
| OursProtocol=Finetuned, Visual-Text PT=VideoCC3M, # Caps=970K2022.04 | 76.9 | 35.8 | 65.1 | — | — | — | |
| BridgeFormerE2E=true, PT Dataset=CC3M, WebVid2M, # Pairs=5.5M, # Frames=42022.12 | 75.1 | 37.6 | 64.8 | 3 | — | — | |
| TT-CE2022.07 | 74.2 | 29.6 | 61.6 | 3 | — | — | |
| ST-AdapterType=Adapter, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=14.222024.05 | 74.2 | 39.5 | 65.1 | — | 20 | — | |
| VideoDistillPre-training set=WebVid, HD-VILA, EgoCLIP, Pre-training pairs=11M, Contrastive fine-tuning=true2024.04 | 74 | 32.8 | 63.5 | — | — | — | |
| MACE2E=true, PT Dataset=CC3M, WebVid2M, # Pairs=5.5M, # Frames=42022.12 | 73.9 | 38.9 | 63.1 | 3 | — | — | |
| VPTType=Prompt, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=0.212024.05 | 73.9 | 37.5 | 63 | — | 21.6 | — | |
| MILESE2E=true, PT Dataset=CC3M, WebVid2M, # Pairs=5.5M, # Frames=42022.12 | 73.8 | 37.7 | 63.6 | 3 | — | — | |
| CoOpType=Prompt, Backbone=CLIP-ViT-B/32, Trainable Params (MB)=0.022024.05 | 73.4 | 38.3 | 62.3 | — | 18.9 | — | |
| COTS# PT Pairs=15.3M, extra_modalities=false2022.04 | 73.2 | 36.8 | 63.8 | — | 2 | — | |
| HIT-pretrained2022.07 | 73.2 | 30.7 | 60.9 | 2.6 | — | — | |
| HIT-pretrainedType=Others2021.11 | 73.2 | 30.7 | 60.9 | 2.6 | — | — | |
| HiTE2E=false, PT Dataset=HowTo100M, # Pairs=120M, # Frames=322022.12 | 73.2 | 30.7 | 60.9 | 2.6 | — | — | |
| ALPROE2E=false, PT Dataset=CC3M, WebVid2M, # Pairs=5.5M, # Frames=82022.12 | 73.2 | 33.9 | 60.7 | 3 | — | — | |
| VIOLETE2E=true, PT Dataset=CC3M, WebVid2M, YT180M, # Pairs=185.5M, # Frames=42022.12 | 73.2 | 34.5 | 63 | — | — | — | |
| HIT-pretrainedTest-set=1k-A2021.06 | 73.2 | 30.7 | 60.9 | 2.6 | — | — | |
| VideoDistillPre-training set=WebVid, HD-VILA, EgoCLIP, Pre-training pairs=11M, Contrastive fine-tuning=false2024.04 | 72.9 | 33.4 | 70.1 | — | — | — | |
| COTS# PT Pairs=5.3M, extra_modalities=false2022.04 | 72.8 | 33.1 | 61.3 | — | 3 | — | |
| FlorencePre-training Type=Image, Pre-training Data=FLD-900M, Evaluation Protocol=Zero-shot2021.11 | 72.6 | 37.6 | 63.8 | — | — | — | |
| FlorencePre-training Type=Image, Pre-training Data=FLD-900M, Evaluation Protocol=Zero-shot2021.11 | 72.6 | 37.6 | 63.8 | — | — | — | |
| FrozenE2E=true, PT Dataset=CC3M, WebVid2M, # Pairs=5.5M, # Frames=42022.12 | 71.6 | 34.2 | 61.1 | 3 | — | — | |
| FITProtocol=Finetuned, Visual-Text PT=Multiple, # Caps=6.1M, Pretraining datasets=WebVid-2M, CC3M and COCO2022.04 | 71.2 | 32.5 | 61.5 | — | — | — | |
| Frozen in TimeE2E=true, Vis Enc. Init.=ImageNet, Visual-Text PT=CC3M,WV-2M,COCO, Pretraining Pairs=6.1M, Zero-shot=false2021.04 | 71.2 | 32.5 | 61.5 | 3 | — | — | |
| TACO# PT Pairs=>100M, extra_modalities=true2022.04 | 71.2 | 28.4 | 57.8 | — | 4 | — | |
| TACOE2E=false, PT Dataset=HowTo100M, # Pairs=120M, # Frames=482022.12 | 71.2 | 28.4 | 57.8 | 4 | — | — | |
| FrozenInTimePre-training set=CC3M, WV2M, COCO, Pre-training pairs=6.1M2024.04 | 71.2 | 32.5 | 61.5 | — | — | — | |
| Frozen in TimeE2E=true, Vis Enc. Init.=ImageNet, Visual-Text PT=CC3M,WV-2M, Pretraining Pairs=5.5M, Zero-shot=false2021.04 | 70.5 | 31 | 59.5 | 3 | — | — |