Video-to-Text Retrieval on MSR-VTT 9K
47.7R@1CenterCLIP
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CenterCLIPMeM. GB=17.6, Speed ms=86.5, Backbone=ViT-B/16, Clustering=k-medoids++, Budget=B6 - 4, 1602022.05 | 47.7 | 75 | 83.3 | 2 | 10.2 | |
| TEFALBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 47.1 | 75.1 | 84.9 | 2 | 7.4 | |
| X-CLIPBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 46.8 | 73.3 | 84 | 2 | 9.1 | |
| TS2-NetBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 45.3 | 74.1 | 83.7 | 2 | 9.2 | |
| CAMOEBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 45.1 | 72.4 | 83.1 | 2 | 10 | |
| BridgeFormerBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 44.9 | 71.9 | 80.3 | 2 | 15.3 | |
| ECLIPSE_meanPBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 44.7 | 71.3 | 82.8 | 2 | 10.8 | |
| VoPF+CParams (M)=14.12022.11 | 44.5 | 70.7 | 80.6 | 2 | 11.5 | |
| X-Pool2022.03 | 44.4 | 73.3 | 84 | 2 | 9 | |
| X-PoolBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 44.4 | 73.3 | 84 | 2 | 9 | |
| VoPF+PParams (M)=0.42022.11 | 43.6 | 71.2 | 81.2 | 2 | 11 | |
| CenterCLIPMeM. GB=14.9, Speed ms=40.8, Backbone=ViT-B/32, Clustering=spectral, Budget=B6 - 4, 492022.05 | 43.5 | 72.1 | 82.2 | 2 | 11.1 | |
| CLIP4clip (meanP)MeM. GB=25.7, Speed ms=59.6, Backbone=ViT-B/16, variant=baseline2022.05 | 43.2 | 72.5 | 80.7 | 2 | 10.9 | |
| CLIP4Clipaggregation=meanP2022.03 | 43.1 | 70.5 | 81.2 | 2 | 12.4 | |
| CLIP4clip (meanP)MeM. GB=20.8, Speed ms=24.42022.05 | 43.1 | 70.5 | 81.2 | 2 | 12.4 | |
| CLIP4clip (meanP)MeM. GB=20.8, Speed ms=24.4, Backbone=ViT-B/32, variant=baseline2022.05 | 43.1 | 70.8 | 80.6 | 2 | 11.4 | |
| CLIP4Clip_meanPBackbone=CLIP ViT-B/32, Pre-training=Wikipedia-based image-text dataset2023.07 | 43.1 | 70.5 | 81.2 | 2 | 12.4 | |
| CenterCLIPMeM. GB=15, Speed ms=22.9, Backbone=ViT-B/32, Clustering=k-medoids++, Budget=B6 - 4, 492022.05 | 42.9 | 70.4 | 80.8 | 2 | 10.8 | |
| CenterCLIPMeM. GB=14.2, Speed ms=22.9, Backbone=ViT-B/32, Clustering=k-medoids++, Budget=B6 - 3, 492022.05 | 42.9 | 71.4 | 81.7 | 2 | 11.1 | |
| CenterCLIPMeM. GB=14.2, Speed ms=43.6, Backbone=ViT-B/32, Clustering=spectral, Budget=B6 - 3, 492022.05 | 42.8 | 71.7 | 82.2 | 2 | 10.9 | |
| CLIP4Clipaggregation=seqTransf2022.03 | 42.7 | 70.9 | 80.6 | 2 | 11.6 | |
| CLIP4clip (seqTransf)2022.05 | 42.7 | 70.9 | 80.6 | 2 | 11.6 | |
| FullParams (M)=119.82022.11 | 42.5 | 70.9 | 81.4 | 2 | 11 | |
| VoPPParams (M)=0.52022.11 | 42.5 | 70 | 79.9 | 2 | 12.4 | |
| VoPFParams (M)=0.12022.11 | 42.4 | 70.5 | 81 | 2 | 11 | |
| VoPCParams (M)=14.32022.11 | 42.3 | 70.1 | 81.1 | 2 | 11.4 | |
| VoPParams (M)=0.12022.11 | 42.1 | 68.8 | 80.7 | 2 | 12.4 | |
| BiasParams (M)=0.12022.11 | 41.1 | 68.4 | 79.2 | 2 | 13.6 | |
| AdapterFFNParams (M)=2.02022.11 | 39.9 | 66.8 | 77.7 | 2 | 14.2 | |
| AdapterATTNParams (M)=2.02022.11 | 39.6 | 66.5 | 76.8 | 2 | 14.7 | |
| PartialParams (M)=7.72022.11 | 37.9 | 66.1 | 77.4 | 3 | 15.5 | |
| ProjParams (M)=0.72022.11 | 37.2 | 64.6 | 75.9 | 3 | 16.7 | |
| TeachText-CE+2022.03 | 32.1 | 62.7 | 75 | 3 | — | |
| TT-CE+2022.05 | 32.1 | 62.7 | 75 | 3 | — | |
| T2VLAD2022.05 | 31.8 | 60 | 71.1 | 3 | — | |
| Support Set2022.03 | 28.5 | 58.6 | 71.6 | 3 | — | |
| AVLnet2022.05 | 28.5 | 58.6 | 71.6 | 3 | — | |
| Straight-CLIP2022.03 | 27.2 | 51.7 | 62.6 | 5 | — | |
| CLIP zero-shot2022.05 | 27.2 | 51.7 | 62.6 | 5 | — | |
| MMT2022.03 | 27 | 57.5 | 69.7 | 3.7 | 21.3 | |
| MMT2022.05 | 27 | 57.5 | 69.7 | 3.7 | 21.3 | |
| CE2022.03 | 20.6 | 50.3 | 64 | 5.3 | 25.1 | |
| CE2022.05 | 20.6 | 50.3 | 64 | 5.3 | — |