Action Recognition on Something-Else
74.5Top-1 AccuracyObjectViViT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ObjectViViTn. FLOPs=1.06x, Temporal views=2, Spatial crops=3, Token sampling percentage=50%2023.06 | 74.5 | — | |
| MFormer w. ObjectLearnern. FLOPs=1.04x, Temporal views=1, Spatial crops=32023.06 | 73.6 | — | |
| ObjectViViTn. FLOPs=1.06x, Temporal views=1, Spatial crops=32023.06 | 72 | — | |
| VideoMAE w. ORVIT (reproduced)n. FLOPs=1.06x, Temporal views=1, Spatial crops=32023.06 | 71.6 | — | |
| MFormer w. ORVITn. FLOPs=1.09x, Temporal views=1, Spatial crops=32023.06 | 69.7 | — | |
| CACNFModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Oracle2025.12 | 67.1 | 90.4 | |
| VideoMAE (our baseline)n. FLOPs=1x (181), Temporal views=1, Spatial crops=32023.06 | 67 | — | |
| MFormern. FLOPs=1x (370), Temporal views=1, Spatial crops=32023.06 | 62.8 | — | |
| SFIModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Oracle2025.12 | 59.6 | 85.8 | |
| StudentBackbone=Swin-T, Input Modality=RGB, lambda=1, gamma=302025.12 | 59.1 | 86.1 | |
| STLTModality=Object detections-based, Evaluation Protocol=Oracle2025.12 | 59 | 86 | |
| OmnivoreBackbone=Swin-T, Input Modality=RGB2025.12 | 58.3 | 84.9 | |
| CACNFModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Obj. predictions2025.12 | 56.9 | 82.5 | |
| STIN + I3DModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Oracle2025.12 | 54.6 | 79.4 | |
| STRGModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Obj. predictions2025.12 | 52.3 | 78.3 | |
| STINModality=Object detections-based, Evaluation Protocol=Oracle2025.12 | 51.4 | 79.3 | |
| STIN + I3DModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Obj. predictions2025.12 | 48.2 | 72.6 | |
| I3DModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Obj. predictions2025.12 | 46.8 | 72.2 | |
| I3DModality=RGB-based and multimodal fusion-based, Evaluation Protocol=Oracle2025.12 | 46.8 | 72.2 | |
| SFIModality=Object detections-based, Evaluation Protocol=Oracle2025.12 | 44.1 | 74 | |
| STLTModality=Object detections-based, Evaluation Protocol=Obj. predictions2025.12 | 41.6 | 67.9 | |
| STINModality=Object detections-based, Evaluation Protocol=Obj. predictions2025.12 | 37.2 | 62.4 |