Action Recognition on MiniSS zero-shot v2
68.8Top-1 AccuracyMAE-Align w/ real humans
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MAE-Align w/ real humansPrivacy Preserving=false, Stage 1: MAE=Kinetics, Stage 2: Alignment=Kinetics, Evaluation Protocol=Fine-tuning, Backbone=ViT-B2023.11 | 68.8 | — | |
| Privacy-Preserving MAE-Align (PPMA)Privacy Preserving=true, Stage 1: MAE=NH Kinetics, Stage 2: Alignment=NH Kinetics + Synthetic, Evaluation Protocol=Fine-tuning, Backbone=ViT-B2023.11 | 67.8 | — | |
| MAE-Align w/ SyntheticPrivacy Preserving=true, Stage 1: MAE=Synthetic, Stage 2: Alignment=Synthetic, Evaluation Protocol=Fine-tuning, Backbone=ViT-B2023.11 | 64.3 | — | |
| R(2+1)D (RN50 backbone)Privacy Preserving=true, Stage 2: Alignment=Synthetic, Evaluation Protocol=Fine-tuning, Backbone=ResNet-502023.11 | 52 | — | |
| TimeSformer SyntheticPrivacy Preserving=false, Stage 2: Alignment=Synthetic, Evaluation Protocol=Fine-tuning, Backbone=ViT-B, Pre-trained on ImageNet-21K=true2023.11 | 51.1 | — | |
| I3D (RN50 backbone)Privacy Preserving=true, Stage 2: Alignment=Synthetic, Evaluation Protocol=Fine-tuning, Backbone=ResNet-502023.11 | 50.7 | — | |
| TSN (RN50 backbone)Privacy Preserving=true, Stage 2: Alignment=Synthetic, Evaluation Protocol=Fine-tuning, Backbone=ResNet-502023.11 | 49.7 | — | |
| TimeSformer KineticsPrivacy Preserving=false, Stage 2: Alignment=Kinetics, Evaluation Protocol=Fine-tuning, Backbone=ViT-B, Pre-trained on ImageNet-21K=true2023.11 | 48.9 | — | |
| MAE-Align w/ real humansPrivacy Preserving=false, Stage 1: MAE=Kinetics, Stage 2: Alignment=Kinetics, Evaluation Protocol=Linear Probing, Backbone=ViT-B2023.11 | 37.4 | — | |
| Privacy-Preserving MAE-Align (PPMA)Privacy Preserving=true, Stage 1: MAE=NH Kinetics, Stage 2: Alignment=NH Kinetics + Synthetic, Evaluation Protocol=Linear Probing, Backbone=ViT-B2023.11 | 34.9 | — | |
| MAE-Align w/ SyntheticPrivacy Preserving=true, Stage 1: MAE=Synthetic, Stage 2: Alignment=Synthetic, Evaluation Protocol=Linear Probing, Backbone=ViT-B2023.11 | 26 | — | |
| TimeSformer KineticsPrivacy Preserving=false, Stage 2: Alignment=Kinetics, Evaluation Protocol=Linear Probing, Backbone=ViT-B, Pre-trained on ImageNet-21K=true2023.11 | 21.5 | — | |
| TimeSformer SyntheticPrivacy Preserving=false, Stage 2: Alignment=Synthetic, Evaluation Protocol=Linear Probing, Backbone=ViT-B, Pre-trained on ImageNet-21K=true2023.11 | 21.2 | — | |
| R(2+1)D (RN50 backbone)Privacy Preserving=true, Stage 2: Alignment=Synthetic, Evaluation Protocol=Linear Probing, Backbone=ResNet-502023.11 | 13.3 | — | |
| TSN (RN50 backbone)Privacy Preserving=true, Stage 2: Alignment=Synthetic, Evaluation Protocol=Linear Probing, Backbone=ResNet-502023.11 | 12.8 | — | |
| I3D (RN50 backbone)Privacy Preserving=true, Stage 2: Alignment=Synthetic, Evaluation Protocol=Linear Probing, Backbone=ResNet-502023.11 | 12.3 | — | |
| ScratchPrivacy Preserving=false, Evaluation Protocol=Fine-tuning, Backbone=ViT-B2023.11 | 9.3 | — | |
| MAXIgt=no, language=K400 dict., GPT3 verbs, BLIP verbs, Zero-shot=true2023.03 | 6.37 | 18.73 | |
| ViFi-CLIPgt=yes, language=K400 dict., Zero-shot=true2023.03 | 5.98 | 19.04 | |
| MAXIgt=no, language=K400 dict., GPT3 verbs, Zero-shot=true2023.03 | 5.6 | 16.73 | |
| MAXIgt=no, language=K400 dict., Zero-shot=true2023.03 | 5.19 | 17.71 | |
| CLIPgt=no, Zero-shot=true2023.03 | 3.96 | 14.42 |