Segment-level Video-to-Music Retrieval on MusicVid-YT8M (test)
7Median RankMVPt (t = 11s)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MVPt (t = 11s)Features=CLIP & DeepSim, Temporal Context=Transformers, Loss Function=InfoNCE, Segment duration (t)=11s2022.06 | 7 | 28.97 | 47.71 | 65.82 | |
| MVPtFeatures=CLIP & DeepSim, Temporal Context=Transformers, Loss Function=InfoNCE2022.06 | 19 | 17.33 | 29.12 | 39.33 | |
| Baseline + CLIP & DeepSim + Visual-time TransformersFeatures=CLIP & DeepSim, Temporal Context=Transformers (visual time)2022.06 | 24 | 17.23 | 27.54 | 38.64 | |
| Baseline + CLIP & DeepSim + Music-time TransformersFeatures=CLIP & DeepSim, Temporal Context=Transformers (music time)2022.06 | 27 | 16.53 | 27.04 | 37.14 | |
| MVPt + X3D featuresFeatures=X3D, Temporal Context=Transformers, Loss Function=InfoNCE2022.06 | 28 | 8.47 | 19.66 | 28.87 | |
| MVPt + TimeSformer featuresFeatures=TimeSformer, Temporal Context=Transformers, Loss Function=InfoNCE2022.06 | 40 | 6.81 | 16.87 | 25.83 | |
| MVPt (t = 4.5s)Features=CLIP & DeepSim, Temporal Context=Transformers, Loss Function=InfoNCE, Segment duration (t)=4.5s2022.06 | 52 | 11.7 | 17.18 | 24.79 | |
| Baseline + CLIP & DeepSim featuresFeatures=CLIP & DeepSim2022.06 | 176 | 3.06 | 6.31 | 9.69 | |
| Baseline2022.06 | 349 | 0.55 | 2.16 | 3.83 | |
| Chance2022.06 | 1,000 | 0.05 | 0.25 | 0.5 |