Action Dynamics on Ego4D and SSv2
92AA (Ego4d)Human
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Humanvariant=Reference baseline2023.05 | 92 | 78 | 96 | 90 | 89 | |
| PAXION (KP-Perceiver)Backbone=InternVideo, Patcher Training Loss=VTC+DVDM, Trainable Param#=4.2M (0.9%)2023.05 | 90.1 | 75.5 | 90.7 | 87.4 | 85.9 | |
| PAXION (KP-Perceiver)Backbone=Clip-ViP, Patcher Training Loss=VTC+DVDM, Trainable Param#=2.4M (1.6%)2023.05 | 89.3 | 56.9 | 89.3 | 66 | 75.4 | |
| PAXION (KP-Perceiver)Backbone=Singularity, Patcher Training Loss=VTC+DVDM, Trainable Param#=1.3M (0.6%)2023.05 | 83.8 | 58.9 | 82.4 | 68.8 | 73.5 | |
| KP-TransformerBackbone=InternVideo, Patcher Training Loss=VTC, Trainable Param#=8.4M (1.8%)2023.05 | 68.2 | 62.8 | 65.5 | 60.6 | 64.3 | |
| KP-PerceiverBackbone=InternVideo, Patcher Training Loss=VTC, Trainable Param#=4.2M (0.9%)2023.05 | 66.5 | 63.6 | 69.8 | 71 | 67.7 | |
| KP-TransformerBackbone=Clip-ViP, Patcher Training Loss=VTC, Trainable Param#=3.9M (2.6%)2023.05 | 61.9 | 53.4 | 72.2 | 54.3 | 60.5 | |
| KP-PerceiverBackbone=Clip-ViP, Patcher Training Loss=VTC, Trainable Param#=2.4M (1.6%)2023.05 | 61.9 | 54.6 | 71.5 | 48.8 | 59.2 | |
| KP-TransformerBackbone=Singularity, Patcher Training Loss=VTC, Trainable Param#=3.9M (1.8%)2023.05 | 61.9 | 48.2 | 63.8 | 49.5 | 55.9 | |
| KP-PerceiverBackbone=Singularity, Patcher Training Loss=VTC, Trainable Param#=1.3M (0.6%)2023.05 | 60.3 | 46.1 | 63.3 | 51.5 | 55.3 | |
| InternVideovariant=Backbone only2023.05 | 58.8 | 46.2 | 51.8 | 48.3 | 51.3 | |
| Clip-ViPvariant=Backbone only2023.05 | 49.3 | 55 | 70.2 | 53.6 | 57 | |
| Singularityvariant=Backbone only2023.05 | 47 | 50.1 | 48.9 | 49.6 | 48.9 |