Text-to-Video Retrieval on MSR-VTT 1K (val)
53.3R@1Weighted Token-wise Interaction (WTI)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Weighted Token-wise Interaction (WTI)Backbone=ViT-B/16, Normalization=QB-Norm2022.03 | 53.3 | 80.3 | 87.6 | 1 | — | |
| Cap4VideoBackbone=CLIP-ViT-B/162022.12 | 51.4 | 75.7 | 83.9 | 1 | 12.4 | |
| Weighted Token-wise Interaction (WTI)Backbone=ViT-B/162022.03 | 50.2 | 76.5 | 84.7 | 1 | — | |
| DRLVenue=arXiv'22, Backbone=CLIP-ViT-B/162022.12 | 50.2 | 76.5 | 84.7 | 1 | — | |
| TS2-NetVenue=ECCV'22, Backbone=CLIP-ViT-B/162022.12 | 49.4 | 75.6 | 85.3 | 2 | 13.5 | |
| Cap4VideoBackbone=CLIP-ViT-B/322022.12 | 49.3 | 74.3 | 83.8 | 2 | 12 | |
| CenterCLIPVenue=SIGIR'22, Backbone=CLIP-ViT-B/162022.12 | 48.4 | 73.8 | 82 | 2 | 13.8 | |
| CLIP2TVVenue=arXiv'21, Backbone=CLIP-ViT-B/162022.12 | 48.3 | 74.6 | 82.8 | 2 | 14.9 | |
| Weighted Token-wise Interaction (WTI)2022.03 | 47.4 | 74.6 | 83.8 | 2 | — | |
| DRLVenue=arXiv'22, Backbone=CLIP-ViT-B/322022.12 | 47.4 | 74.6 | 83.8 | 2 | — | |
| QB-NormVenue=CVPR'22, Backbone=CLIP-ViT-B/322022.12 | 47.2 | 73 | 83 | 2 | — | |
| TS2-NetVenue=ECCV'22, Backbone=CLIP-ViT-B/322022.12 | 47 | 74.5 | 83.8 | 2 | 13 | |
| X-PoolVenue=CVPR'22, Backbone=CLIP-ViT-B/322022.12 | 46.9 | 72.8 | 82.2 | 2 | 14.3 | |
| CLIP2VideoVenue=arXiv'21, Backbone=CLIP-ViT-B/322022.12 | 45.6 | 72.6 | 81.7 | 2 | 14.6 | |
| CAMOEVenue=arXiv'21, Backbone=CLIP-ViT-B/322022.12 | 44.6 | 72.6 | 81.8 | 2 | 13.3 | |
| CLIP4Clip2022.03 | 44.5 | 71.4 | 81.6 | 2 | — | |
| CLIP4ClipVenue=arXiv'21, Backbone=CLIP-ViT-B/322022.12 | 44.5 | 71.4 | 81.6 | 2 | 15.3 | |
| CenterCLIPVenue=SIGIR'22, Backbone=CLIP-ViT-B/322022.12 | 44.2 | 71.6 | 82.1 | 2 | 15.1 | |
| WebVid2MVideo-Text Training Data=WebVid2M [4], Evaluation Protocol=Zero-shot2023.10 | 38.5 | 61.7 | 71.9 | 3 | — | |
| BridgeFormerVenue=CVPR'222022.12 | 37.6 | 64.8 | 75.1 | — | — | |
| HowToCaptionVideo-Text Training Data=HowToCaption (ours), Evaluation Protocol=Zero-shot2023.10 | 37.6 | 62 | 73.3 | 3 | — | |
| TMVMVenue=NeurIPS'222022.12 | 36.2 | 64.2 | 75.7 | 3 | — | |
| BLIP InitializationVideo-Text Training Data=None (BLIP initialization), Evaluation Protocol=Zero-shot2023.10 | 34.3 | 59.8 | 70.6 | 3 | — | |
| VideoCC3MVideo-Text Training Data=VideoCC3M [39], Evaluation Protocol=Zero-shot2023.10 | 33.9 | 57.9 | 67.1 | 4 | — | |
| FrozenVenue=ICCV'212022.12 | 32.5 | 61.5 | 71.2 | 3 | — | |
| FROZEN2022.03 | 31 | 59.5 | 70.5 | 3 | — | |
| HowTo100M (ASRs)Video-Text Training Data=HowTo100M with ASRs [17], Evaluation Protocol=Zero-shot2023.10 | 30.8 | 52.6 | 61.7 | 5 | — | |
| SupportSetVenue=ICLR'212022.12 | 30.1 | 58.5 | 69.3 | 3 | — | |
| HTM-AAVideo-Text Training Data=HTM-AA (auto-aligned ASRs) [17], Evaluation Protocol=Zero-shot2023.10 | 29.8 | 54.1 | 64.3 | 4 | — | |
| T2VLADVenue=CVPR'212022.12 | 29.5 | 59 | 70.1 | 4 | — | |
| HowTo100M (distant supervision)Video-Text Training Data=HowTo100M with distant supervision [31], Evaluation Protocol=Zero-shot2023.10 | 28.6 | 54 | 66.3 | 4 | — | |
| SUPPORT2022.03 | 27.4 | 56.3 | 67.7 | 3 | — | |
| MDMMT2022.03 | 26.6 | 57.1 | 69.6 | 4 | — | |
| MMTVenue=ECCV'202022.12 | 26.6 | 57.1 | 69.6 | 4 | — | |
| ClipBERT2022.03 | 22 | 46.8 | 59.9 | 6 | — | |
| ClipBERTVenue=CVPR'202022.12 | 22 | 46.8 | 59.9 | 6 | — | |
| UniVL2022.03 | 21.2 | 49.6 | 63.1 | 6 | — | |
| HERO2022.03 | 16.8 | 43.4 | 57.7 | — | — |