Text-to-Video Retrieval on Youcook2 (test)
98.9Recall@10VideoCLIP
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| VideoCLIPMeasure=DTW, Setting=Zero-shot2024.01 | 98.9 | 55.7 | 93.1 | — | — | |
| VideoCLIPMeasure=OTAM, Setting=Zero-shot2024.01 | 98.9 | 56.6 | 92.8 | — | — | |
| VideoCLIPMeasure=Cap. Avg., Setting=Zero-shot2024.01 | 98.4 | 73.6 | 94.7 | — | — | |
| NortonMeasure=Cap. Avg., Setting=Zero-shot2024.01 | 98.4 | 74.8 | 94.7 | — | — | |
| TempCLRMeasure=Cap. Avg., Setting=Zero-shot2024.01 | 97.9 | 71.7 | 94.5 | — | — | |
| TempCLRMeasure=DTW, Setting=Zero-shot2024.01 | 97.9 | 70.4 | 93.8 | — | — | |
| NortonMeasure=DTW, Setting=Zero-shot2024.01 | 97.7 | 76.1 | 95 | — | — | |
| TempCLRMeasure=OTAM, Setting=Zero-shot2024.01 | 97.7 | 72.2 | 94.5 | — | — | |
| NortonMeasure=OTAM, Setting=Zero-shot2024.01 | 97.7 | 73.6 | 94.7 | — | — | |
| VAST + OursIncorporate Audio and Subtitles=true2026.03 | 84.5 | 53 | 77.1 | — | — | |
| VASTIncorporate Audio and Subtitles=false2026.03 | 80.8 | 50.4 | 74.3 | — | — | |
| VideoCLIPEvaluation Protocol=Supervised, mode=Fine-tuned2021.09 | 75 | 32.2 | 62.6 | — | — | |
| MDMMT-22022.03 | 74.8 | 32 | 64 | 3 | 12.7 | |
| MELTRIncorporate Audio and Subtitles=true2026.03 | 74.8 | 33.7 | 63.1 | — | — | |
| TACo2022.03 | 72.7 | 29.6 | 59.7 | 4 | — | |
| UniVL2022.03 | 70 | 28.9 | 57.6 | 4 | — | |
| UniVLIncorporate Audio and Subtitles=true2026.03 | 70 | 28.9 | 57.6 | — | — | |
| VLMIncorporate Audio and Subtitles=false2026.03 | 69.4 | 27.1 | 56.9 | — | — | |
| UniVL (FT-Joint)Evaluation Protocol=Supervised2021.09 | 66.2 | 22.2 | 52.2 | — | — | |
| TempCLRFeature=S3D-G2024.01 | 64.5 | 23.3 | 51 | — | — | |
| NortonFeature=S3D-G2024.01 | 64.1 | 24.2 | 51.9 | — | — | |
| VideoCLIPEvaluation Protocol=Zero-shot2021.09 | 63.1 | 22.7 | 50.4 | — | — | |
| VideoCLIPFeature=S3D-G2024.01 | 63.1 | 22.7 | 50.4 | — | — | |
| TANFeature=S3D-G2024.01 | 59.5 | 20.1 | 45.5 | — | — | |
| TACOFeature=S3D-G2024.01 | 55.7 | 19.9 | 43.2 | — | — | |
| MMFTFeature=S3D-G2024.01 | 55.1 | 19.8 | 42.9 | — | — | |
| VAST, HowToCaption-finetunedV. Encoder=ViT-G2023.10 | 53.9 | 19.7 | 43.6 | 8 | — | |
| CootEvaluation Protocol=Supervised2021.09 | 52.3 | 16.7 | 40.2 | — | — | |
| COOT2022.03 | 52.3 | 16.7 | — | — | — | |
| MIL-NCEEvaluation Protocol=Zero-shot2021.09 | 51.2 | 15.1 | 38 | — | — | |
| MIL-NCEBatch size=8192, Epoch=27, Evaluation Protocol=Zero-shot2021.04 | 51.2 | — | — | 10 | — | |
| MIL-NCEFeature=S3D-G2024.01 | 51.2 | 15.1 | 38 | — | — | |
| BLIP*, HowToCaption-finetunedV. Encoder=ViT-B2023.10 | 50.4 | 18.15 | 38.68 | 10 | — | |
| VAST†V. Encoder=ViT-G2023.10 | 45.6 | 15.7 | 35.2 | 14 | — | |
| VATT-MBSBatch size=2048, Epoch=4, Evaluation Protocol=Zero-shot2021.04 | 45.5 | — | — | 13 | — | |
| MMVBatch size=4096, Epoch=8, Evaluation Protocol=Zero-shot2021.04 | 45.4 | — | — | 13 | — | |
| MCNFeature=R152+RX1012024.01 | 45.2 | 18.1 | 35.5 | — | — | |
| HowToCaptionVision Encoder=ViT-B/16, Image-Text Data=6 datasets, Video-Text Data=HowToCaption2023.10 | 44.1 | 13.4 | 33.1 | 15 | — | |
| VATT-MA-MediumBatch size=2048, Epoch=4, Evaluation Protocol=Zero-shot2021.04 | 40.6 | — | — | 17 | — | |
| VT-TWINS2022.03 | 40.3 | 9.7 | 28 | 16 | — | |
| BLIP + HowTo100MVision Encoder=ViT-B/16, Image-Text Data=6 datasets, Video-Text Data=HowTo100M2023.10 | 39.3 | 12.2 | 29.1 | 20 | — | |
| VT-TWINSFeature=S3D-G2024.01 | 38.8 | 9.7 | 27 | — | — | |
| VT-TWINSLabeled Dataset=None2022.03 | 38.8 | 9.7 | 27 | 19 | — | |
| ActBERTFeature=R101+Res3D2024.01 | 38 | 9.6 | 26.7 | — | — | |
| ActBERTLabeled Dataset=YC22022.03 | 38 | 9.6 | 26.7 | 19 | — | |
| MIL-NCEimplementation=reproduction of official code2022.03 | 37.36 | 9.35 | 26.22 | 22 | — | |
| Text-Video Embedding2022.03 | 35.3 | 8.2 | 24.5 | 24 | — | |
| Miech et al.Labeled Dataset=IM, K400, YC22022.03 | 35.3 | 8.2 | 24.5 | 24 | — | |
| MIL-NCELabeled Dataset=None2022.03 | 34.6 | 8.8 | 24.3 | 23 | — | |
| BLIP*V. Encoder=ViT-B2023.10 | 32.3 | 10.7 | 24.1 | 44.5 | — | |
| HowTo100MEvaluation Protocol=Zero-shot2021.09 | 24.8 | 6.1 | 17.3 | — | — | |
| Miech et al.Labeled Dataset=IM, K4002022.03 | 24.8 | 6.1 | 17.3 | 46 | — | |
| COOTLabeled Dataset=YC22022.03 | 24.8 | 5.9 | 16.7 | 49.7 | — | |
| BLIPVision Encoder=ViT-B/16, Image-Text Data=6 datasets2023.10 | 23.6 | 6.1 | 16.2 | 69 | — | |
| HGLMMEvaluation Protocol=Supervised2021.09 | 21.6 | 4.6 | 14.3 | — | — | |
| HGLMM FC CCALabeled Dataset=IM, K400, YC22022.03 | 21.6 | 4.6 | 14.3 | 75 | — | |
| RandomEvaluation Protocol=Zero-shot2021.09 | 0.3 | 0 | 0.2 | — | — | |
| Random InitLabeled Dataset=None2022.03 | 0.3 | 0.03 | 0.15 | 1,675 | — | |
| Random Init2022.03 | 0.26 | 0.03 | 0.13 | 1,717.5 | — |