Long-form Video Understanding on LVU (test)
67.11Relation Top-1 AccS5 + LSMCL
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| S5 + LSMCLNumber of input frames=100, Pre-training=LSMCL, GPU Usage (GB)=3.952023.03 | 67.11 | 42.12 | 73.49 | 67.32 | 51.27 | 47.95 | 65.41 | 0.24 | 3.51 | — | — | |
| S5Number of input frames=100, GPU Usage (GB)=3.952023.03 | 66.71 | 41.78 | 73.28 | 66.64 | 50.6 | 47.85 | 63.65 | 0.25 | 3.51 | — | — | |
| S5Number of input frames=60, GPU Usage (GB)=3.852023.03 | 61.98 | 41.75 | 69.88 | 66.4 | 50.6 | 47.7 | 58.8 | 0.25 | 3.51 | — | — | |
| S5 + LSMCLNumber of input frames=60, Pre-training=LSMCL, GPU Usage (GB)=3.852023.03 | 61.98 | 41.75 | 72.53 | 66.4 | 50.6 | 47.7 | 61.34 | 0.24 | 3.51 | — | — | |
| ViS4merGPU Usage (GB)=5.152023.03 | 57.14 | 40.79 | 67.44 | 62.61 | 48.8 | 44.75 | 54.71 | 0.26 | 3.63 | — | — | |
| Obj. T4merGPU Usage (GB)=N/A2023.03 | 54.76 | 33.17 | 52.94 | 47.66 | 36.3 | 37.76 | 52.74 | 0.3 | 3.68 | — | — | |
| VideoBERTGPU Usage (GB)=N/A2023.03 | 52.8 | 37.9 | 54.9 | 47.3 | 38.5 | 36.1 | 51.9 | 0.32 | 4.46 | — | — | |
| Supervised Scene (Sup)instance_representation=None, scene_representation=Sup, scene_pretraining=K400 fully-supervised, backbone=Slow-D2022.04 | 52.38 | 34.37 | 26.51 | 23.18 | 5.36 | 18.88 | 47.37 | 0.595 | 4.061 | — | 4.44 | |
| Hierarchical SSL (Scene only)instance_representation=None, scene_representation=Ours, scene_pretraining=CVRL + TxE (event mask prediction), backbone=Slow-D2022.04 | 52.38 | 33.07 | 36.98 | 42.43 | 23.93 | 35.24 | 48.11 | 0.375 | 4.653 | — | 2.89 | |
| LSTGPU Usage (GB)=41.382023.03 | 52.38 | 37.31 | 62.79 | 56.07 | 42.26 | 39.16 | 52.7 | 0.31 | 3.83 | — | — | |
| CVRLinstance_representation=CVRL, scene_representation=None, backbone=R502022.04 | 50.95 | 32.86 | 32.56 | 37.76 | 27.26 | 25.31 | 48.17 | 0.444 | 4.6 | — | 3.89 | |
| OT++instance_representation=OT, scene_representation=Ours, scene_pretraining=CVRL + TxE (event mask prediction), backbone=Slow-D2022.04 | 50.95 | 34.07 | 44.19 | 40.19 | 31.43 | 29.65 | 51.15 | 0.353 | 4.886 | — | 2.33 | |
| OTinstance_representation=OT, scene_representation=None, backbone=R50, temporal_network=Slow-D, pretraining_dataset=10k movies2022.04 | 50 | 34.57 | 32.56 | 37.38 | 26.55 | 25.73 | 49.55 | 0.396 | 4.559 | — | 3.22 | |
| PerformerGPU Usage (GB)=5.932023.03 | 50 | 38.8 | 60.46 | 58.87 | 48.21 | 41.25 | 49.45 | 0.31 | 3.93 | — | — | |
| OrthoformerGPU Usage (GB)=5.562023.03 | 50 | 38.3 | 66.27 | 55.14 | 47.02 | 43.35 | 55.79 | 0.29 | 3.86 | — | — | |
| MoDistinstance_representation=MoDist, scene_representation=None, backbone=R502022.04 | 49.52 | 33.57 | 30.7 | 40.56 | 23.1 | 26.57 | 49.26 | 0.458 | 4.506 | — | 3.89 |