Video-Text Retrieval on DIDEMO
44.5GFLOPSvid-TLDR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| vid-TLDRbackbone=UMT-B, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 44.5 | 42.3 | 68.4 | 77 | |
| vid-TLDRbackbone=UMT-B, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 47.3 | 35.9 | 60.6 | 69 | |
| vid-TLDRbackbone=UMT-B, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 48.2 | 44 | 68.9 | 77.7 | |
| UMT + ToMebackbone=UMT-B, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 62.8 | 38.6 | 64.7 | 73.8 | |
| UMT + ToMebackbone=UMT-B, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 62.8 | 38.6 | 64.5 | 73 | |
| UMT + ToMebackbone=UMT-B, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 66.2 | 29.7 | 55.5 | 65.1 | |
| UMT (Base)backbone=UMT-B, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 78.5 | 35.8 | 61.9 | 70.1 | |
| UMT (Base)backbone=UMT-B, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 78.5 | 41.1 | 66.7 | 75.4 | |
| UMT (Base)backbone=UMT-B, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 78.5 | 41 | 66.6 | 75.8 | |
| vid-TLDRbackbone=UMT-L, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 162.5 | 52 | 74.9 | 82.4 | |
| vid-TLDRbackbone=UMT-L, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 166.2 | 36.6 | 62.1 | 71 | |
| vid-TLDRbackbone=UMT-L, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 166.7 | 48.7 | 74.3 | 81.3 | |
| UMT + ToMebackbone=UMT-L, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 182.1 | 33.7 | 60 | 69.5 | |
| UMT + ToMebackbone=UMT-L, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 182.1 | 45.4 | 71.8 | 80.1 | |
| UMT + ToMebackbone=UMT-L, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 182.1 | 47.9 | 72.6 | 81.1 | |
| UMT (Base)backbone=UMT-L, pretraining_pairs=5M, eval_protocol=zero-shot2024.03 | 267.8 | 35.1 | 60.2 | 68.7 | |
| UMT (Base)backbone=UMT-L, pretraining_pairs=17M, eval_protocol=zero-shot2024.03 | 267.8 | 46.5 | 71.1 | 79.2 | |
| UMT (Base)backbone=UMT-L, pretraining_pairs=25M, eval_protocol=zero-shot2024.03 | 267.8 | 49.3 | 73.9 | 80.2 |