Multi-Instance Retrieval on EPIC-KITCHENS-100
45mAP (Avg)Frozen + EgoNCE
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Frozen + EgoNCEVis Enc Input=Raw Videos, #Frames=16, Vis-text PT=EgoClip, Evaluation Protocol=Fine-tuning2022.06 | 45 | 49.9 | 40.1 | 60.9 | 57.9 | 59.4 | |
| JPOSEVis Enc Input=TBN † [14], #Frames=25, Evaluation Protocol=Fine-tuning2022.06 | 44 | 49.9 | 38.1 | 55.5 | 51.6 | 53.5 | |
| FrozenVis Enc Input=Raw Videos, #Frames=16, Vis-text PT=CC3M+WebVid-2M, Evaluation Protocol=Fine-tuning2022.06 | 40.9 | 45.8 | 36 | 57.2 | 54.3 | 55.8 | |
| Frozen + EgoNCEVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=EgoClip, Evaluation Protocol=Fine-tuning2022.06 | 40.2 | 45.1 | 35.3 | 56.2 | 53.5 | 54.8 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=EgoClip, Evaluation Protocol=Fine-tuning2022.06 | 39.6 | 44.5 | 34.7 | 55.7 | 52.9 | 54.3 | |
| MMEVis Enc Input=TBN † [14], #Frames=25, Evaluation Protocol=Fine-tuning2022.06 | 38.5 | 43 | 34 | 50.1 | 46.9 | 48.5 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=CC3M+WebVid-2M, Evaluation Protocol=Fine-tuning2022.06 | 36.4 | 41.2 | 31.6 | 52.7 | 49.7 | 51.4 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=HowTo100M, Evaluation Protocol=Fine-tuning2022.06 | 34.7 | 39.2 | 30.1 | 50.7 | 48.7 | 49.7 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Evaluation Protocol=Fine-tuning2022.06 | 34.2 | 38.8 | 29.7 | 50.5 | 48.3 | 49.4 | |
| MI-MMVis Enc Input=S3D [42], #Frames=32, Vis-text PT=HowTo100M, Evaluation Protocol=Fine-tuning2022.06 | 29.2 | 34.8 | 23.6 | 47.1 | 42.4 | 44.7 | |
| HierVL-SAEvaluation Protocol=Zero-shot2023.01 | 18.9 | — | — | — | — | 24.7 | |
| HierVL-w/o HierEvaluation Protocol=Zero-shot2023.01 | 17.8 | — | — | — | — | 24.1 | |
| HierVL-AvgEvaluation Protocol=Zero-shot2023.01 | 16.7 | — | — | — | — | 23.5 | |
| Frozen + EgoNCEVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=EgoClip, Evaluation Protocol=Zero-shot2022.06 | 16.6 | 19.4 | 13.9 | 24.1 | 22 | 23.1 | |
| EgoVLPEvaluation Protocol=Zero-shot2023.01 | 16.6 | — | — | — | — | 23.1 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=EgoClip, Evaluation Protocol=Zero-shot2022.06 | 15.5 | 17.9 | 13.1 | 23 | 21.2 | 22.1 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=CC3M+WebVid-2M, Evaluation Protocol=Zero-shot2022.06 | 8 | 8.6 | 7.4 | 14.5 | 14.6 | 14.5 | |
| FrozenVis Enc Input=Raw Videos, #Frames=4, Vis-text PT=HowTo100M, Evaluation Protocol=Zero-shot2022.06 | 6.5 | 6.8 | 6.3 | 11.6 | 12.8 | 12.2 | |
| Random2022.06 | 5.7 | 5.7 | 5.6 | 10.8 | 10.9 | 10.9 |