Video-Text Retrieval on MSRVTT
44.7GFLOPSvid-TLDR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| vid-TLDRbackbone=UMT-B, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 44.7 | 33.3 | 54.7 | 33.3 | |
| vid-TLDRbackbone=UMT-B, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 50.2 | 27.7 | 49.8 | 58.3 | |
| vid-TLDRbackbone=UMT-B, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 50.6 | 33.8 | 57 | 66.6 | |
| UMT + ToMebackbone=UMT-B, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 66.2 | 26.4 | 46.7 | 55.9 | |
| UMT + ToMebackbone=UMT-B, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 66.2 | 32.5 | 54.1 | 63.9 | |
| UMT + ToMebackbone=UMT-B, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 66.2 | 31.1 | 52.8 | 31.1 | |
| UMT (Base)backbone=UMT-B, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 78.5 | 27.9 | 49.8 | 58.4 | |
| UMT (Base)backbone=UMT-B, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 78.5 | 33.6 | 56.4 | 66.4 | |
| UMT (Base)backbone=UMT-B, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 78.5 | 32.8 | 54.3 | 32.8 | |
| vid-TLDRbackbone=UMT-L, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 146.5 | 39.2 | 61.2 | 39.2 | |
| UMT + ToMebackbone=UMT-L, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 160.9 | 36.3 | 59 | 36.3 | |
| vid-TLDRbackbone=UMT-L, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 163 | 39.9 | 61.9 | 70.9 | |
| vid-TLDRbackbone=UMT-L, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 164.8 | 32.7 | 55.1 | 64.6 | |
| UMT + ToMebackbone=UMT-L, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 182.1 | 30.4 | 52.4 | 62.9 | |
| UMT + ToMebackbone=UMT-L, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 182.1 | 37.6 | 60.1 | 68.9 | |
| UMT (Base)backbone=UMT-L, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 267.8 | 31.8 | 54.7 | 64.2 | |
| UMT (Base)backbone=UMT-L, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 267.8 | 40.6 | 62 | 71.4 | |
| UMT (Base)backbone=UMT-L, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 267.8 | 38.9 | 61.1 | 38.9 |