Action Anticipation on Epic-Kitchen 55 (val)
22.7Top-1 AccS-GEAR
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| S-GEARModalities=RGB+Obj+Flow2024.07 | 22.7 | 43.2 | — | — | — | — | — | — | — | |
| Our modelBackbone=Resnet18(2D+1D), Temporal gap (dt)=1.0s, Input modalities=rgb+optical flow+object, Losses=JVS, JCC, JFIP2021.05 | 20.35 | 39.15 | — | — | — | — | — | — | — | |
| AVHNTemporal gap (dt)=1.0s2021.05 | 19.29 | 35.91 | — | — | — | — | — | — | — | |
| Ego-OMGModalities=RGB+HOI+NAO2024.07 | 19.2 | — | — | — | — | — | — | — | — | |
| DCRModalities=RGB+Obj+Flow2024.07 | 19.2 | 41.2 | — | — | — | — | — | — | — | |
| AVT+Modalities=RGB+Obj2024.07 | 16.6 | 37.6 | — | — | — | — | — | — | — | |
| AGABackbone=Swin-B, Resolution=224x2242026.03 | 16.3 | 37.4 | — | — | — | — | — | — | 16.6 | |
| S-GEARModality=RGB, Encoder=irCSN, Initialization=IG65M2024.07 | 16.2 | 33.1 | — | — | — | — | — | — | — | |
| S-GEARBackbone=irCSN1522026.03 | 16.2 | 33.1 | — | — | — | — | — | — | — | |
| DCRModality=RGB, Encoder=TSM, Initialization=K4002024.07 | 16.1 | 33.1 | — | — | — | — | — | — | — | |
| DCRBackbone=TSM2026.03 | 16.1 | 33.1 | — | — | — | — | — | — | — | |
| S-GEARModality=RGB, Encoder=ViT-B, Initialization=IN21K2024.07 | 15.8 | 34.5 | — | — | — | — | — | — | — | |
| S-GEARBackbone=ViT-B2026.03 | 15.8 | 34.5 | — | — | — | — | — | — | — | |
| S-GEARModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | 15.6 | 32.8 | — | — | — | — | — | — | — | |
| S-GEARBackbone=TSN, Resolution=384x3842026.03 | 15.6 | 32.8 | — | — | — | — | — | — | — | |
| RU-LSTMModalities=RGB+Obj+Flow2024.07 | 15.3 | 35.3 | — | — | — | — | — | — | — | |
| ImaginationModalities=RGB+Obj+Flow2024.07 | 15.2 | 35.4 | — | — | — | — | — | — | — | |
| TempAgg.Modalities=RGB+Obj+Flow2024.07 | 15.1 | 35.6 | — | — | — | — | — | — | — | |
| DCRBackbone=irCSN1522026.03 | 15.1 | 34 | — | — | — | — | — | — | — | |
| AVTModality=RGB, Encoder=irCSN, Initialization=IG65M2024.07 | 14.4 | 31.7 | — | — | — | — | — | — | — | |
| DCRModality=RGB, Encoder=irCSN, Initialization=IG65M2024.07 | 14.4 | 34 | — | — | — | — | — | — | — | |
| AVT-hBackbone=irCSN1522026.03 | 14.4 | 31.7 | — | — | — | — | — | — | 13.2 | |
| FIATemporal gap (dt)=1.0s2021.05 | 14.07 | 33.37 | — | — | — | — | — | — | — | |
| RAFTformerBackbone=TSN2026.03 | 13.8 | — | — | — | — | — | — | — | — | |
| ImagineRNNBackbone=TSN2026.03 | 13.7 | 31.6 | — | — | — | — | — | — | — | |
| DCRModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | 13.6 | 30.8 | — | — | — | — | — | — | — | |
| DCRBackbone=TSN2026.03 | 13.6 | 30.8 | — | — | — | — | — | — | — | |
| AGABackbone=TSN, Resolution=224x2242026.03 | 13.5 | 32.1 | — | — | — | — | — | — | 14.3 | |
| RULSTMbackbone=TSN2022.05 | 13.1 | — | — | — | — | — | — | — | — | |
| AVTbackbone=TSN2022.05 | 13.1 | — | — | — | — | — | — | — | — | |
| RU LSTMModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | 13.1 | 30.8 | — | — | — | — | — | — | — | |
| AVTModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | 13.1 | 28.1 | — | — | — | — | — | — | — | |
| RULSTMBackbone=TSN2026.03 | 13.1 | 30.8 | — | — | — | — | — | — | 12.5 | |
| AVT-hBackbone=TSN2026.03 | 13.1 | 28.1 | — | — | — | — | — | — | 13.5 | |
| AVTModality=RGB, Encoder=ViT-B, Initialization=IN21K2024.07 | 12.5 | 30.1 | — | — | — | — | — | — | — | |
| AVT-hBackbone=AVT-b2026.03 | 12.5 | 30.1 | — | — | — | — | — | — | 13.6 | |
| S-GEARModality=Obj, Encoder=FRCNN, Initialization=IN1K2024.07 | 12.45 | 30.4 | — | — | — | — | — | — | — | |
| Temporal Agg.backbone=TSN2022.05 | 12.3 | — | — | — | — | — | — | — | — | |
| FUTRbackbone=TSN2022.05 | 12.3 | — | — | — | — | — | — | — | — | |
| TempAggBackbone=TSN2026.03 | 12.3 | 28.5 | — | — | — | — | — | — | 13.1 | |
| DCRModality=Obj, Encoder=FRCNN, Initialization=IN1K2024.07 | 11.5 | 30.5 | — | — | — | — | — | — | — | |
| SVM-TOP3Temporal gap (dt)=1.0s2021.05 | 11.09 | 25.42 | — | — | — | — | — | — | — | |
| S-GEARModality=Flow, Encoder=TSN, Initialization=IN1K2024.07 | 10.8 | 25.8 | — | — | — | — | — | — | — | |
| BaselineBackbone=Resnet18(2D+1D), Temporal gap (dt)=1.0s, Input modalities=rgb+optical flow+object2021.05 | 10.44 | 25.63 | — | — | — | — | — | — | — | |
| RULSTMModality=Obj, Encoder=FRCNN, Initialization=IN1K2024.07 | 10 | 29.8 | — | — | — | — | — | — | — | |
| DCRModality=Flow, Encoder=TSN, Initialization=IN1K2024.07 | 8.9 | 22.7 | — | — | — | — | — | — | — | |
| RULSTMModality=Flow, Encoder=TSN, Initialization=IN1K2024.07 | 8.7 | 21.4 | — | — | — | — | — | — | — | |
| VN-CETemporal gap (dt)=1.0s2021.05 | 5.79 | 17.31 | — | — | — | — | — | — | — | |
| ActionBanks2021.06 | — | — | 35.8 | 80 | 23.4 | 52.8 | 15.1 | 35.6 | — | |
| ATSN2021.06 | — | — | — | 77.3 | — | 39.9 | — | 16.3 | — | |
| AVT+backbone=irCSN-152, late fusion=true2021.06 | — | — | 32.5 | 79.9 | 24.4 | 54 | 16.6 | 37.6 | — | |
| DMR2021.06 | — | — | — | 73.7 | — | 30 | — | 16.9 | — | |
| ED2021.06 | — | — | — | 75.5 | — | 43 | — | 25.8 | — | |
| EL2021.06 | — | — | — | 75.7 | — | 43.7 | — | 28.6 | — | |
| FHOIbackbone=I3D2021.06 | — | — | 30.7 | 76.5 | 17.4 | 42.6 | 10.4 | 25.5 | — | |
| FN2021.06 | — | — | — | 74.8 | — | 40.9 | — | 26.3 | — | |
| HROModalities=RGB+Obj+Flow2024.07 | — | 37.4 | — | — | — | — | — | — | — | |
| ImagineRNN2021.06 | — | — | — | — | — | — | — | 35.6 | — | |
| MCE2021.06 | — | — | — | 73.4 | — | 38.9 | — | 26.1 | — | |
| RLTemporal gap (dt)=1.0s2021.05 | — | 29.61 | — | — | — | — | — | — | — | |
| RL2021.06 | — | — | — | 76.8 | — | 44.5 | — | 29.6 | — | |
| RU LSTMTemporal gap (dt)=1.0s2021.05 | — | 35.32 | — | — | — | — | — | — | — | |
| RULSTM2021.06 | — | — | 32.4 | 79.6 | 23.5 | 51.8 | 15.3 | 35.3 | — | |
| SRLModality=RGB, Encoder=TSN, Initialization=IN1K2024.07 | — | 31.7 | — | — | — | — | — | — | — | |
| SRLModalities=RGB+Obj+Flow2024.07 | — | 35.5 | — | — | — | — | — | — | — | |
| SRLBackbone=TSN2026.03 | — | 31.7 | — | — | — | — | — | — | 13.2 |