Action Anticipation on Epic-Kitchens-100 (val)
52.54mCR@5 (Overall Verb)InAVIT
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| InAVIT2022.11 | 52.54 | — | — | — | 51.93 | 25.89 | 46.45 | 51.3 | 25.33 | 45.34 | 39.21 | 20.22 | — | |
| MFtrained_by_authors=true2022.11 | 47.14 | — | — | — | 46.92 | 21.52 | 42.33 | 47.22 | 21.99 | 40.47 | 38.66 | 16.36 | — | |
| ORVIT-MFtrained_by_authors=true2022.11 | 46.71 | — | — | — | 47.91 | 23.54 | 38.99 | 45.32 | 23.6 | 37.28 | 37.81 | 17.1 | — | |
| TrAc2022.11 | 35.04 | — | — | — | 35.49 | 16.6 | 34.64 | 27.26 | 13.83 | 30.08 | 33.64 | 15.53 | — | |
| Rformer2022.11 | 33.8 | — | — | — | 37.9 | 19.1 | — | — | — | — | — | — | — | |
| RAFTformer-2BBackbone=MViTv2-16 + 242026.03 | 33.8 | — | — | — | 37.9 | 19.1 | — | — | — | — | — | — | — | |
| RAFTformerBackbone=MViTv2-162026.03 | 33.3 | — | — | — | 35.5 | 17.6 | — | — | — | — | — | — | — | |
| MeMViT, 16×4External data / extra annotations=K400, Params (M)=592022.01 | 32.8 | — | — | — | 33.2 | 15.1 | 27.5 | 21.7 | 9.8 | 26.3 | 27.4 | 13.2 | — | |
| MeMViT 16x4Backbone=MViTv2-162026.03 | 32.8 | — | — | — | 33.2 | 15.1 | 27.5 | 21.7 | 9.8 | 26.3 | 27.4 | 13.2 | — | |
| S-GEAR-2BBackbone=ViT-B x22026.03 | 32.7 | — | — | — | 37.9 | 19.6 | — | — | — | — | — | — | — | |
| AGABackbone=Swin-B2026.03 | 32.5 | — | — | — | 38.7 | 18.8 | 34.4 | 28.5 | 16.3 | 27.4 | 35 | 18.4 | — | |
| MeMVITModalities=RGB only, Backbone training protocol=frozen2022.10 | 32.2 | — | — | — | 37 | 17.7 | 28.6 | 27.4 | 15.2 | 25.3 | 31 | 15.5 | — | |
| MeMViT, 32×3External data / extra annotations=K700, Params (M)=2122022.01 | 32.2 | — | — | — | 37 | 17.7 | 28.6 | 27.4 | 15.2 | 25.3 | 31 | 15.5 | — | |
| MeMViT2022.11 | 32.2 | — | — | — | 37 | 17.7 | 28.6 | 27.4 | 15.2 | 25.3 | 31 | 15.5 | — | |
| MeMViT 32x3Backbone=MViTv2-242026.03 | 32.2 | — | — | — | 37 | 17.7 | 28.6 | 27.4 | 15.2 | 25.3 | 31 | 15.5 | — | |
| AGABackbone=TSN2026.03 | 32.2 | — | — | — | 35.7 | 17.5 | 31.4 | 24.9 | 11.9 | 26.9 | 31.8 | 16.8 | — | |
| S-GEARBackbone=ViT-B2026.03 | 31.1 | — | — | — | 37.3 | 18.3 | — | — | — | — | — | — | — | |
| AVT (RGB)External data / extra annotations=IN21K, Params (M)=3782022.01 | 30.2 | — | — | — | 31.7 | 14.9 | — | — | — | — | — | — | — | |
| AVT-hBackbone=AVT-b2026.03 | 30.2 | — | — | — | 31.7 | 14.9 | — | — | — | — | — | — | — | |
| AVT+modalities=late fuse2021.06 | 28.2 | — | — | — | 32 | 15.9 | 29.5 | 23.9 | 11.9 | 21.1 | 25.8 | 14.1 | — | |
| AVT+Backbone training protocol=frozen2022.10 | 28.2 | — | — | — | 32 | 15.9 | 29.5 | 23.9 | 11.9 | 21.1 | 25.8 | 14.1 | — | |
| AVT+ (RGB + Obj)External data / extra annotations=IN21K + EPIC boxes2022.01 | 28.2 | — | — | — | 32 | 15.9 | 29.5 | 23.9 | 11.9 | 21.1 | 25.8 | 14.1 | — | |
| AVT2022.11 | 28.2 | — | — | — | 32 | 15.9 | 29.5 | 23.9 | 11.9 | 21.1 | 25.8 | 14.1 | — | |
| RULSTMmodalities=all (RGB, Flow, OBJ)2021.06 | 27.8 | — | — | — | 30.8 | 14 | 28.8 | 27.2 | 14.2 | 19.8 | 22 | 11.1 | — | |
| RULSTMModalities=all, Backbone training protocol=frozen2022.10 | 27.8 | — | — | — | 30.8 | 14 | 28.8 | 27.2 | 14.2 | 19.8 | 22 | 11.1 | — | |
| RULSTM (RGB + Obj + Flow)External data / extra annotations=IN1K + EPIC boxes2022.01 | 27.8 | — | — | — | 30.8 | 14 | 28.8 | 27.2 | 14.2 | 19.8 | 22 | 11.1 | — | |
| T. Agg.2022.11 | 27.8 | — | — | — | 30.8 | 14 | 28.8 | 27.2 | 14.2 | 19.8 | 22 | 11.1 | — | |
| RULSTMBackbone=TSN2026.03 | 27.5 | — | — | — | 29 | 13.3 | — | — | — | — | — | — | — | |
| AVT-hBackbone=TSN2026.03 | 27.2 | — | — | — | 30.7 | 13.6 | — | — | — | — | — | — | — | |
| S-GEARBackbone=TSN2026.03 | 25.8 | — | — | — | 29.8 | 14.9 | — | — | — | — | — | — | — | |
| AVT+backbone=TSN, modalities=late fuse2021.06 | 25.5 | — | — | — | 31.8 | 14.8 | 25.5 | 23.6 | 11.5 | 18.5 | 25.8 | 12.6 | — | |
| AVT+-TSNBackbone=TSN, Backbone training protocol=frozen2022.10 | 25.5 | — | — | — | 31.8 | 14.8 | 25.5 | 23.6 | 11.5 | 18.5 | 25.8 | 12.6 | — | |
| TSN-AVT+ (RGB + Obj)External data / extra annotations=IN21K + EPIC boxes2022.01 | 25.5 | — | — | — | 31.8 | 14.8 | 25.5 | 23.6 | 11.5 | 18.5 | 25.8 | 12.6 | — | |
| AVT-hBackbone=irCSN1522026.03 | 25.5 | — | — | — | 28.1 | 12.8 | — | — | — | — | — | — | — | |
| TempAgg (RGB)External data / extra annotations=IN1K2022.01 | 24.2 | — | — | — | 29.8 | 13 | 27 | 23 | 12.2 | 16.2 | 22.9 | 10.4 | — | |
| AFFT-SwinBackbone=Swin, Observation Time=16s, Backbone training protocol=frozen2022.10 | 23.4 | — | — | — | 33.7 | 17.6 | 24.5 | 25.4 | 15.2 | 15.6 | 26.5 | 15.3 | — | |
| AFFTBackbone=Swin-B2026.03 | 23.4 | — | — | — | 33.7 | 17.6 | 24.5 | 25.4 | 15.2 | 15.6 | 26.5 | 15.3 | — | |
| TempAggModalities=all + hand-object + audio, Backbone training protocol=frozen2022.10 | 23.2 | — | — | — | 31.4 | 14.7 | 28 | 26.2 | 14.5 | 14.5 | 22.5 | 11.8 | — | |
| TempAgg (RGB + Obj + Flow + ROI)External data / extra annotations=IN1K + EPIC boxes2022.01 | 23.2 | — | — | — | 31.4 | 14.7 | 28 | 26.2 | 14.5 | 14.5 | 22.5 | 11.8 | — | |
| RU-LSTM2022.11 | 23.2 | — | — | — | 31.4 | 14.7 | 28 | 26.2 | 14.5 | 14.5 | 22.5 | 11.8 | — | |
| AFFT-Swin+Backbone=Swin, Modalities=additional hand-object + audio, Observation Time=16s, Backbone training protocol=frozen2022.10 | 22.8 | — | — | — | 34.6 | 18.5 | 24.8 | 26.4 | 15.5 | 15 | 27.7 | 16.2 | — | |
| AFFT2022.11 | 22.8 | — | — | — | 34.6 | 18.5 | 24.8 | 26.4 | 15.5 | 15 | 27.7 | 16.2 | — | |
| AFFT-TSN+Backbone=TSN, Modalities=additional hand-object + audio, Observation Time=18s, Backbone training protocol=frozen2022.10 | 22.3 | — | — | — | 31.5 | 17 | 23.8 | 25.3 | 14 | 14.6 | 23.6 | 15 | — | |
| AFFT-TSNBackbone=TSN, Observation Time=18s, Backbone training protocol=frozen2022.10 | 21.3 | — | — | — | 32.7 | 16.4 | 24.1 | 25.5 | 13.6 | 13.2 | 25.8 | 14.3 | — | |
| AFFTBackbone=TSN2026.03 | 21.3 | — | — | — | 32.7 | 16.4 | 24.1 | 25.5 | 13.6 | 13.2 | 25.8 | 14.3 | — | |
| chance2021.06 | 6.4 | — | — | — | 2 | 0.2 | 14.4 | 2.9 | 0.5 | 1.6 | 0.2 | 0.1 | — | |
| chance2022.10 | 6.4 | — | — | — | 2 | 0.2 | 14.4 | 2.9 | 0.5 | 1.6 | 0.2 | 0.1 | — | |
| chance2022.01 | 6.4 | — | — | — | 2 | 0.2 | 14.4 | 2.9 | 0.5 | 1.6 | 0.2 | 0.1 | — | |
| AFFTModalities=RGB+Obj+Flow+HOI+Audio2024.07 | — | 22.8 | 34.6 | 18.5 | — | — | — | — | — | — | — | — | — | |
| AVTModality=RGB, Encoder=ViT, Initialization=IN21K2024.07 | — | 30.2 | 31.7 | 14.9 | — | — | — | — | — | — | — | — | — | |
| AVTModality=Obj, Encoder=FRCNN, Initialization=IN1K2024.07 | — | 18 | 24.3 | 8.7 | — | — | — | — | — | — | — | — | — | |
| AVTModality=Flow, Encoder=TSN, Initialization=IN1K2024.07 | — | 20.9 | 16.9 | 6.6 | — | — | — | — | — | — | — | — | — | |
| AVT+Modalities=RGB+Obj2024.07 | — | 28.2 | 32 | 15.9 | — | — | — | — | — | — | — | — | — | |
| DCRModality=RGB, Encoder=TSM, Initialization=K4002024.07 | — | 32.6 | 32.7 | 16.1 | — | — | — | — | — | — | — | — | — | |
| DCRModality=Obj, Encoder=FRCNN, Initialization=IN1K2024.07 | — | 22.2 | 24.2 | 9.7 | — | — | — | — | — | — | — | — | — | |
| DCRModality=Flow, Encoder=TSN, Initialization=IN1K2024.07 | — | 25.9 | 17.6 | 8.4 | — | — | — | — | — | — | — | — | — | |
| InAViTParams=160M2026.05 | — | 51.9 | 52 | 25.8 | — | — | — | — | — | — | — | — | — | |
| InAVITParameters=160M2025.06 | — | 51.9 | 52 | 25.8 | — | — | — | — | — | — | — | — | — | |
| MeMViTModality=RGB, Encoder=MViTv2-16, Initialization=K4002024.07 | — | 32.8 | 33.2 | 15.1 | — | — | — | — | — | — | — | — | — | |
| MeMViTModality=RGB, Encoder=MViTv2-24, Initialization=K7002024.07 | — | 32.2 | 37 | 17.7 | — | — | — | — | — | — | — | — | — | |
| PlausiVLParameters=8B2025.06 | — | 55.6 | 54.2 | 27.6 | — | — | — | — | — | — | — | — | — | |
| PlausiVLParams=8B2026.05 | — | 55.6 | 54.2 | 27.6 | — | — | — | — | — | — | — | — | — | |
| RAFTformerModality=RGB, Encoder=MViTv2-16, Initialization=K4002024.07 | — | 33.7 | 37.1 | 18 | — | — | — | — | — | — | — | — | — | |
| RAFTformerModality=RGB, Encoder=MViTv2-24, Initialization=K7002024.07 | — | 33.3 | 35.5 | 17.6 | — | — | — | — | — | — | — | — | — | |
| RAFTformer-2BModality=RGB, Encoder=MViTv2-16&24, Initialization=K400&7002024.07 | — | 33.8 | 37.9 | 19.1 | — | — | — | — | — | — | — | — | — | |
| RU-LSTMModalities=RGB+Obj+Flow2024.07 | — | 27.8 | 30.8 | 14 | — | — | — | — | — | — | — | — | — | |
| RU-LSTMModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | — | — | — | 13.3 | — | — | — | — | — | — | — | — | — | |
| S-GEARModalities=RGB+Obj2024.07 | — | 29.5 | 37.8 | 18.9 | — | — | — | — | — | — | — | — | — | |
| S-GEARModality=RGB, Encoder=ViT-B, Initialization=IN21K2024.07 | — | 31.1 | 37.3 | 18.3 | — | — | — | — | — | — | — | — | — | |
| S-GEARModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | — | 25.8 | 29.8 | 14.9 | — | — | — | — | — | — | — | — | — | |
| S-GEARModality=RGB, Encoder=irCSN, Initialization=IG65M2024.07 | — | 26.8 | 28.8 | 13.3 | — | — | — | — | — | — | — | — | — | |
| S-GEARModality=Obj, Encoder=FRCNN, Initialization=IN1K2024.07 | — | 20.8 | 28.6 | 11.4 | — | — | — | — | — | — | — | — | — | |
| S-GEARModality=Flow, Encoder=TSN, Initialization=IN1K2024.07 | — | 21.5 | 18.2 | 7.9 | — | — | — | — | — | — | — | — | — | |
| S-GEAR-2BModalities=RGB+Obj2024.07 | — | 30.5 | 38.4 | 19.6 | — | — | — | — | — | — | — | — | — | |
| S-GEAR-2BModality=RGB, Encoder=ViT-B×2, Initialization=IN21K2024.07 | — | 32.7 | 37.9 | 19.6 | — | — | — | — | — | — | — | — | — | |
| S-GEAR-4BModalities=RGB+Obj2024.07 | — | 30.2 | 37 | 19.9 | — | — | — | — | — | — | — | — | — | |
| TempAgg.Modalities=RGB+Obj+Flow+HOI+Audio2024.07 | — | 23.2 | 31.4 | 14.7 | — | — | — | — | — | — | — | — | — | |
| TrajPilot (No-Traj)Params=1B, Features=V-JEPA 2.1, Probe=V-JEPA 2, Mode=finetuned2026.05 | — | 60.62 | 56.7 | 38.27 | — | — | — | — | — | — | — | — | 65.42 | |
| TrajPilot (Oracle)Params=1B, Features=V-JEPA 2.1, Probe=V-JEPA 2, Mode=finetuned2026.05 | — | 61.79 | 57.59 | 38.55 | — | — | — | — | — | — | — | — | 65.95 | |
| TrajPilot (Scorer)Params=1B, Features=V-JEPA 2.1, Probe=V-JEPA 2, Mode=finetuned2026.05 | — | 60.74 | 57.23 | 38.21 | — | — | — | — | — | — | — | — | 65.41 | |
| V-JEPA 2 ViT-gParameters=1B, Frozen Backbone=true2025.06 | — | 61.2 | 55.7 | 38 | — | — | — | — | — | — | — | — | — | |
| V-JEPA 2 ViT-gParams=1B2026.05 | — | 63.6 | 57.1 | 39.7 | — | — | — | — | — | — | — | — | — | |
| V-JEPA 2 ViT-g384Parameters=1B, Frozen Backbone=true, Higher Resolution=3842025.06 | — | 63.6 | 57.1 | 39.7 | — | — | — | — | — | — | — | — | — | |
| V-JEPA 2 ViT-HParameters=600M, Frozen Backbone=true2025.06 | — | 59.2 | 54.6 | 36.5 | — | — | — | — | — | — | — | — | — | |
| V-JEPA 2 ViT-LParameters=300M, Frozen Backbone=true2025.06 | — | 57.8 | 53.8 | 32.7 | — | — | — | — | — | — | — | — | — | |
| V-JEPA 2.1 ViT-gParams=1B2026.05 | — | 63.6 | 56.2 | 38.4 | — | — | — | — | — | — | — | — | — | |
| V-JEPA 2.1 ViT-GParams=2B2026.05 | — | 64.3 | 59.9 | 40.8 | — | — | — | — | — | — | — | — | — | |
| Video-LLaMAParameters=7B2025.06 | — | 52.9 | 52 | 26 | — | — | — | — | — | — | — | — | — | |
| Video-LLaMAParams=7B2026.05 | — | 52.9 | 52 | 26 | — | — | — | — | — | — | — | — | — | |
| Visual-only attentive probeParams=1B, Features=V-JEPA 2.1, Probe=V-JEPA 2, Mode=finetuned2026.05 | — | 60.21 | 56.39 | 37.77 | — | — | — | — | — | — | — | — | 64.07 |