Action Anticipation on EPIC-Kitchens-100 (test)
27.6Overall Action Top-5 RecallPlausiVL
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PlausiVL2024.05 | 27.6 | 55.62 | 54.23 | — | — | — | — | — | — | — | — | — | |
| Video LLAMA2024.05 | 26.05 | 52.9 | 52.01 | — | — | — | — | — | — | — | — | — | |
| InAVIT2024.05 | 25.89 | 52.54 | 51.93 | — | — | — | — | — | — | — | — | — | |
| InAVIT2022.11 | 23.75 | 49.14 | 49.97 | 44.36 | 49.28 | 23.49 | 43.17 | 39.91 | 18.11 | — | — | — | |
| ORVIT-MFtrained_by_authors=true2022.11 | 21.53 | 43.74 | 46.61 | 38.99 | 45.32 | 21.47 | 37.28 | 35.78 | 15.96 | — | — | — | |
| SCUTchallenge_entry=ensemble2022.10 | 20.4 | 37.9 | 41.7 | 27.9 | 37.1 | 18.3 | 32.4 | 36.1 | 17.1 | — | — | — | |
| MFtrained_by_authors=true2022.11 | 19.75 | 45.14 | 45.97 | 40.36 | 45.28 | 19.49 | 39.17 | 35.91 | 14.11 | — | — | — | |
| NVIDIA-UNIBZchallenge_entry=ensemble2022.10 | 19.6 | 29.7 | 38.5 | 23.5 | 35.2 | 16.4 | 23.5 | 31.1 | 16.6 | — | — | — | |
| ICL-SJTUchallenge_entry=ensemble2022.10 | 19.5 | 42 | 35.7 | 33.4 | 26.8 | 15.9 | 41 | 33.2 | 16.9 | — | — | — | |
| MATModality=RGB + OF, Init=IN-1K2023.08 | 19.5 | 35 | 38.8 | — | — | — | — | — | — | — | — | — | |
| RAFTformer2024.05 | 19.1 | 33.8 | 37.9 | — | — | — | — | — | — | — | — | — | |
| MATModality=RGB, Init=IN-1K2023.08 | 18.8 | 32.7 | 39.7 | — | — | — | — | — | — | — | — | — | |
| PCO-PSNRDchallenge_entry=ensemble2022.10 | 18.7 | 30.9 | 41.3 | 25.7 | 35.4 | 16.3 | 25 | 35.4 | 16.1 | — | — | — | |
| AFFT2024.05 | 18.5 | 22.8 | 34.6 | — | — | — | — | — | — | — | — | — | |
| MeMViT2024.05 | 17.7 | 32.2 | 37 | — | — | — | — | — | — | — | — | — | |
| TeSTraModality=RGB + OF, Init=IN-1K2023.08 | 17.6 | 30.8 | 35.8 | — | — | — | — | — | — | — | — | — | |
| allenxuuuchallenge_entry=ensemble2022.10 | 17.4 | 29.9 | 30.4 | 25.1 | 26.1 | 14.1 | 24.6 | 23.7 | 14.3 | — | — | — | |
| TeSTraModality=RGB, Init=IN-1K2023.08 | 17 | 26.8 | 36.2 | — | — | — | — | — | — | — | — | — | |
| AGAModality=RGB, Backbone=Swin-B2026.03 | 16.9 | 30.8 | 36.4 | 22.3 | 30 | 13.5 | 25.8 | 30 | 14.9 | — | — | — | |
| AVT2022.11 | 16.74 | 26.69 | 32.33 | 21.03 | 27.64 | 12.89 | 19.28 | 24.03 | 13.81 | — | — | — | |
| AVT++description=ensembles multiple model variants2021.06 | 16.7 | 26.7 | 32.3 | 21 | 27.6 | 12.9 | 19.3 | 24 | 13.8 | — | — | — | |
| AVT++challenge_entry=ensemble2022.10 | 16.7 | 26.7 | 32.3 | 21 | 27.6 | 12.9 | 19.3 | 24 | 13.8 | — | — | — | |
| AVT++Modality=RGB, Flow, Obj2026.03 | 16.7 | 26.7 | 32.3 | 21 | 27.6 | 12.9 | 19.3 | 24 | 13.8 | — | — | — | |
| AVT+Modality=RGB + OF + OBJ, Init=IN-21K2023.08 | 15.9 | 28.2 | 32 | — | — | — | — | — | — | — | — | — | |
| AVT2024.05 | 15.9 | 28.2 | 32 | — | — | — | — | — | — | — | — | — | |
| S-GEAR-4BModality=RGB, Obj2026.03 | 15.5 | 26.6 | 32.6 | — | — | — | — | — | — | — | — | — | |
| Rformer2022.11 | 15.4 | 30.1 | 34.1 | — | — | — | — | — | — | — | — | — | |
| Video LLM2024.05 | 15.4 | — | — | — | — | — | — | — | — | — | — | — | |
| RAFTformer-2BModality=RGB2026.03 | 15.4 | 30.1 | 34.1 | — | — | — | — | — | — | — | — | — | |
| S-GEAR-2BModality=RGB, Obj2026.03 | 15.3 | 25.5 | 31.7 | — | — | — | — | — | — | — | — | — | |
| AFFT-Swin+Backbone=Swin, Modalities=additional hand-object + audio, Observation Time=16s, Backbone training protocol=frozen2022.10 | 14.9 | 20.7 | 31.8 | 16.2 | 27.7 | 12.1 | 13.4 | 23.8 | 11.8 | — | — | — | |
| AVTModality=RGB, Init=IN-21K2023.08 | 14.9 | 30.2 | 31.7 | — | — | — | — | — | — | — | — | — | |
| AFFT2022.11 | 14.9 | 20.7 | 31.8 | 16.2 | 27.7 | 12.1 | 13.4 | 23.8 | 11.8 | — | — | — | |
| AFFT-Swin+Modality=RGB, Flow, Obj, Audio2026.03 | 14.9 | 20.7 | 31.8 | 16.2 | 27.7 | 12.1 | 13.4 | 23.8 | 11.8 | — | — | — | |
| Panasonic2021.06 | 14.8 | 30.4 | 33.5 | 21.1 | 27.1 | 10.2 | 24.6 | 27.5 | 12.7 | — | — | — | |
| AVT+Modality=RGB + OF + OBJ, Init=IN-1K2023.08 | 14.8 | 25.5 | 31.8 | — | — | — | — | — | — | — | — | — | |
| TempAggModality=RGB + OF + OBJ, Init=IN-1K2023.08 | 14.7 | 23.2 | 31.4 | — | — | — | — | — | — | — | — | — | |
| RU-LSTM2024.05 | 14.7 | 23.2 | 31.4 | — | — | — | — | — | — | — | — | — | |
| RAFTformerModality=RGB2026.03 | 14.7 | 27.4 | 34 | — | — | — | — | — | — | — | — | — | |
| S-GEARModality=RGB, Obj2026.03 | 14.7 | 32 | 25.9 | — | — | — | — | — | — | — | — | — | |
| DCRModality=RGB, Init=IN-1K2023.08 | 14.6 | 31 | 31.1 | — | — | — | — | — | — | — | — | — | |
| Abs. goal2022.11 | 14.29 | 31.4 | 30.1 | 31.36 | 35.56 | 17.34 | 22.9 | 16.42 | 7.7 | — | — | — | |
| RULSTMModality=RGB + OF + OBJ, Init=IN-1K2023.08 | 14 | 27.8 | 30.8 | — | — | — | — | — | — | — | — | — | |
| Temporal Aggregation2024.05 | 14 | 27.8 | 30.8 | — | — | — | — | — | — | — | — | — | |
| AVTModality=RGB, Init=IN-1K2023.08 | 13.6 | 27.2 | 30.7 | — | — | — | — | — | — | — | — | — | |
| ICL+SJTU2021.06 | 13.4 | 36.2 | 32.2 | 27.6 | 24.2 | 10.1 | 32.1 | 29.9 | 11.9 | — | — | — | |
| AFFT-TSN+Backbone=TSN, Modalities=additional hand-object + audio, Observation Time=18s, Backbone training protocol=frozen2022.10 | 13.4 | 19.4 | 28.3 | 14 | 24.2 | 9.9 | 12 | 19.5 | 10.9 | — | — | — | |
| AFFT-TSN+Modality=RGB, Flow, Obj, Audio2026.03 | 13.4 | 19.4 | 28.3 | 14 | 24.2 | 9.9 | 12 | 19.5 | 10.9 | — | — | — | |
| TrAct2022.11 | 13.39 | 36.15 | 32.2 | 27.6 | 24.24 | 10.05 | 32.06 | 29.87 | 11.88 | — | — | — | |
| RULSTMModality=RGB, Init=IN-1K2023.08 | 13.3 | 27.5 | 29 | — | — | — | — | — | — | — | — | — | |
| AVT+2021.06 | 12.6 | 25.6 | 28.8 | 20.9 | 22.3 | 8.8 | 19 | 22 | 10.1 | — | — | — | |
| NUS_CVML2021.06 | 12.6 | 21.8 | 30.6 | 17.9 | 27 | 10.5 | 13.6 | 20.6 | 8.9 | — | — | — | |
| AVT+Backbone training protocol=frozen2022.10 | 12.6 | 25.6 | 28.8 | 20.9 | 22.3 | 8.8 | 19 | 22 | 10.1 | — | — | — | |
| TempAggModalities=all + hand-object + audio, Backbone training protocol=frozen2022.10 | 12.6 | 21.8 | 30.6 | 17.9 | 27 | 10.5 | 13.6 | 20.6 | 8.9 | — | — | — | |
| AVT+Modality=RGB, Obj2026.03 | 12.6 | 25.6 | 28.8 | 20.9 | 22.3 | 8.8 | 19 | 22 | 10.1 | — | — | — | |
| T. Agg.2022.11 | 12.55 | 21.76 | 30.59 | 17.86 | 27.04 | 10.46 | 13.59 | 20.62 | 8.85 | — | — | — | |
| RU-LSTMAnticipation time (Ta)=12021.07 | 11.2 | 25.3 | 26.7 | 19.4 | 26.9 | 9.7 | 17.6 | 15.9 | 7.9 | — | — | — | |
| RULSTM2021.06 | 11.2 | 25.3 | 26.7 | 19.4 | 26.9 | 9.7 | 17.6 | 16 | 7.9 | — | — | — | |
| IIE_MRG2021.06 | 11.2 | 25.3 | 26.7 | 19.4 | 26.9 | 9.7 | 17.6 | 16 | 7.9 | — | — | — | |
| RULSTMModalities=all, Backbone training protocol=frozen2022.10 | 11.2 | 25.3 | 26.7 | 19.4 | 26.9 | 9.7 | 17.6 | 16 | 7.9 | — | — | — | |
| RULSTMModality=RGB, Flow, Obj2026.03 | 11.2 | 25.3 | 26.7 | 19.4 | 26.9 | 9.7 | 17.6 | 16 | 7.9 | — | — | — | |
| RU-LSTM2022.11 | 11.19 | 25.25 | 26.69 | 19.36 | 26.87 | 9.65 | 17.56 | 15.97 | 7.92 | — | — | — | |
| Multi-Modal Temporal Convolutional NetworkAnticipation time (Ta)=1, Backbone=TBN2021.07 | 11 | 21.5 | 26.8 | 20.8 | 28.3 | 12.2 | 13.2 | 15.4 | 7.2 | — | — | — | |
| TBN2021.06 | 11 | 21.5 | 26.8 | 20.8 | 28.3 | 12.2 | 13.2 | 15.4 | 7.2 | — | — | — | |
| TCN-TBNBackbone=TBN, Backbone training protocol=frozen2022.10 | 11 | 21.5 | 26.8 | 20.8 | 28.3 | 12.2 | 13.2 | 15.4 | 7.2 | — | — | — | |
| Multi-Modal Temporal Convolutional NetworkAnticipation time (Ta)=1, Backbone=TSN2021.07 | 10.9 | 20.4 | 26.6 | 17.9 | 26.9 | 11.1 | 11.7 | 15.2 | 7 | — | — | — | |
| TCN-TSNBackbone=TSN, Backbone training protocol=frozen2022.10 | 10.9 | 20.4 | 26.6 | 17.9 | 26.9 | 11.1 | 11.7 | 15.2 | 7 | — | — | — | |
| RandomAnticipation time (Ta)=12021.07 | 0.1 | 6.2 | 2.3 | 8.1 | 3.3 | 0.3 | 1.9 | 0.7 | 0.03 | — | — | — | |
| chance2021.06 | 0.1 | 6.2 | 2.3 | 8.1 | 3.3 | 0.3 | 1.9 | 0.7 | 0 | — | — | — | |
| chance2022.10 | 0.1 | 6.2 | 2.3 | 8.1 | 3.3 | 0.3 | 1.9 | 0.7 | 0 | — | — | — | |
| Abstract goal2022.09 | — | — | — | — | — | — | — | — | — | 31.4 | 30.1 | 14.29 | |
| AFFTModalities=RGB+Obj+Flow+HOI+Audio2024.07 | — | — | — | — | — | — | — | — | — | 20.7 | 31.8 | 14.9 | |
| AVT2022.09 | — | — | — | — | — | — | — | — | — | 26.69 | 32.33 | 16.74 | |
| AVTModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 30.2 | 31.7 | 14.9 | |
| AVT+Modalities=RGB+Obj2024.07 | — | — | — | — | — | — | — | — | — | 25.6 | 28.8 | 12.6 | |
| AVT+Modality=RGB+OF+Obj2025.10 | — | — | — | — | — | — | — | — | — | 28.2 | 32 | 15.9 | |
| CPMModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | — | — | 17.2 | |
| CPMModality=RGB+OF+Obj2025.10 | — | — | — | — | — | — | — | — | — | — | — | 19.4 | |
| MATModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 32.7 | 39.7 | 18.8 | |
| MATModality=RGB+OF2025.10 | — | — | — | — | — | — | — | — | — | 35 | 38.8 | 19.5 | |
| MeMViTModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 32.8 | 33.2 | 15.1 | |
| MeMVIT2022.09 | — | — | — | — | — | — | — | — | — | 32.2 | 37 | 17.7 | |
| RU-LSTM2022.09 | — | — | — | — | — | — | — | — | — | 25.25 | 26.69 | 11.19 | |
| RU-LSTMModalities=RGB+Obj+Flow2024.07 | — | — | — | — | — | — | — | — | — | 25.3 | 26.7 | 11.2 | |
| RULSTMModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 27.5 | 29 | 13.3 | |
| RULSTMModality=RGB+OF+Obj2025.10 | — | — | — | — | — | — | — | — | — | 27.8 | 30.8 | 14 | |
| S-GEARModalities=RGB+Obj2024.07 | — | — | — | — | — | — | — | — | — | 25.9 | 32 | 14.7 | |
| S-GEAR-2BModalities=RGB+Obj2024.07 | — | — | — | — | — | — | — | — | — | 25.5 | 31.7 | 15.3 | |
| S-GEAR-2BModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 32.7 | 37.9 | 19.6 | |
| S-GEAR-2BModality=RGB+Obj2025.10 | — | — | — | — | — | — | — | — | — | 30.5 | 38.4 | 19.6 | |
| S-GEAR-4BModalities=RGB+Obj2024.07 | — | — | — | — | — | — | — | — | — | 26.6 | 32.6 | 15.5 | |
| S-GEAR-4BModality=RGB+Obj2025.10 | — | — | — | — | — | — | — | — | — | 30.2 | 37 | 19.9 | |
| State-Specific Model (Ours)Modality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 36.8 | 39.2 | 19.9 | |
| State-Specific Model (Ours)Modality=RGB+OF2025.10 | — | — | — | — | — | — | — | — | — | 38.8 | 42.1 | 21.4 | |
| State-Specific Model (Ours)Modality=RGB+OF+Obj2025.10 | — | — | — | — | — | — | — | — | — | 44.9 | 48.3 | 24.9 | |
| Temp. Agg.2022.09 | — | — | — | — | — | — | — | — | — | 21.76 | 30.59 | 12.55 | |
| TempAgg.Modalities=RGB+Obj+Flow+HOI+Audio2024.07 | — | — | — | — | — | — | — | — | — | 21.8 | 30.8 | 12.6 | |
| TeSTraModality=RGB2025.10 | — | — | — | — | — | — | — | — | — | 26.8 | 36.2 | 17 | |
| TeSTraModality=RGB+OF2025.10 | — | — | — | — | — | — | — | — | — | 30.8 | 35.8 | 17.6 | |
| TransAction2022.09 | — | — | — | — | — | — | — | — | — | 36.15 | 32.2 | 13.39 |