Audio-to-Video Retrieval on MSR-VTT
23.8Recall@1TVLT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TVLTVision Input=true, Audio Input=true, Pretrain Datasets=YTT-S2022.09 | 23.8 | — | — | |
| TVLTVision Input=true, Audio Input=true, Pretrain Datasets=HT100M2022.09 | 22.6 | — | — | |
| AVLnetVision Input=true, Audio Input=true, Pretrain Datasets=HT100M2022.09 | 20.1 | — | — | |
| AVLnetZero-shot=true, Pre-train=HowTo100M2024.03 | 17.8 | 35.5 | 43.6 | |
| EquiAVZero-shot=true2024.03 | 14.4 | 33.3 | 43 | |
| AranjelovicZero-shot=true, Pre-train=HowTo100M2024.03 | 12.6 | 26.3 | 33.7 | |
| VAB-DACDirection=Audio -> Video, Zero-shot=true2024.09 | 12 | 27.3 | 36.2 | |
| VAB-EncodecDirection=Audio -> Video, Zero-shot=true2024.09 | 9.6 | 23.3 | 32.9 | |
| BoggustZero-shot=true, Pre-train=HowTo100M2024.03 | 7.6 | 21.1 | 28.3 | |
| CAV-MAEZero-shot=true2024.03 | 7.6 | 19.8 | 30.2 | |
| CAV-MAEDirection=Audio -> Video, Zero-shot=true2024.09 | 7.6 | 19.8 | 30.2 | |
| ImageBindDirection=Audio -> Video, Zero-shot=true2024.09 | 6 | 16.9 | 23.7 | |
| LanguageBindDirection=Audio -> Video, Zero-shot=true2024.09 | 1.2 | 4.4 | 6.9 |