Segment-level Music-to-Video Retrieval on MusicVid-YT8M (test)
6Median RankMVPt (t = 11s)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MVPt (t = 11s)Features=CLIP & DeepSim, Temporal Context=Transformers, Loss Function=InfoNCE, Segment duration (t)=11s2022.06 | 6 | 29.37 | 47.52 | 65.32 | 65.57 | |
| MVPtFeatures=CLIP & DeepSim, Temporal Context=Transformers, Loss Function=InfoNCE2022.06 | 12 | 19.98 | 34.81 | 45.41 | 42.37 | |
| Baseline + CLIP & DeepSim + Visual-time TransformersFeatures=CLIP & DeepSim, Temporal Context=Transformers (visual time)2022.06 | 24 | 17.07 | 26.85 | 38.43 | 38.54 | |
| Baseline + CLIP & DeepSim + Music-time TransformersFeatures=CLIP & DeepSim, Temporal Context=Transformers (music time)2022.06 | 26 | 16.5 | 26.86 | 37.17 | 37.15 | |
| MVPt + X3D featuresFeatures=X3D, Temporal Context=Transformers, Loss Function=InfoNCE2022.06 | 27 | 8.83 | 19.88 | 29.2 | 29.03 | |
| MVPt + TimeSformer featuresFeatures=TimeSformer, Temporal Context=Transformers, Loss Function=InfoNCE2022.06 | 36 | 7.36 | 17.51 | 26.34 | 26.09 | |
| MVPt (t = 4.5s)Features=CLIP & DeepSim, Temporal Context=Transformers, Loss Function=InfoNCE, Segment duration (t)=4.5s2022.06 | 52 | 10.62 | 16.6 | 24.55 | 24.67 | |
| Baseline + CLIP & DeepSim featuresFeatures=CLIP & DeepSim2022.06 | 107 | 4.71 | 8.14 | 12.14 | 10.91 | |
| Baseline2022.06 | 277 | 0.74 | 2.79 | 4.83 | 4.33 | |
| Chance2022.06 | 1,000 | 0.05 | 0.25 | 0.5 | 0.5 |