Video Action Classification on HMDB51
77.5Top-1 AccuracyPE-G
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PE-GEncoder=PE-G, Protocol=Linear probing2026.02 | 77.5 | 95.54 | |
| X-Ray VisualEncoder=EViT-2b, Protocol=Linear probing2026.02 | 74.69 | 94.21 | |
| VATTPre-training Dataset=HT100M + AudioSet + YT8M2022.11 | 65.84 | 91.43 | |
| VATT + CLPre-training Dataset=HT100M + AudioSet + YT8M, Gradient-based Curriculum Learning (CL)=true2022.11 | 65.77 | 92.15 | |
| DiNO-v3Encoder=ViT-H, Protocol=Linear probing2026.02 | 65.58 | 91.75 | |
| VATT + BothPre-training Dataset=HT100M + AudioSet, Cross-Modality Gradient Realignment (GR)=true, Gradient-based Curriculum Learning (CL)=true2022.11 | 65.52 | 89.74 | |
| VATT + CLPre-training Dataset=HT100M + AudioSet, Gradient-based Curriculum Learning (CL)=true2022.11 | 65.45 | 88.94 | |
| VATT + BothPre-training Dataset=HT100M + AudioSet + YT8M, Cross-Modality Gradient Realignment (GR)=true, Gradient-based Curriculum Learning (CL)=true2022.11 | 64.35 | 92.08 | |
| VATTPre-training Dataset=HT100M + AudioSet2022.11 | 63.1 | — | |
| VATT + RW (VT)Pre-training Dataset=HT100M + AudioSet, Gradient Re-weighting (RW)=Video-Text2022.11 | 62.3 | 86.61 | |
| VATT + GRPre-training Dataset=HT100M + AudioSet, Cross-Modality Gradient Realignment (GR)=true2022.11 | 62.3 | 90.38 | |
| VATT + RW (VT)Pre-training Dataset=HT100M + AudioSet + YT8M, Gradient Re-weighting (RW)=Video-Text2022.11 | 61.13 | 90.51 | |
| VATT + RW (VA)Pre-training Dataset=HT100M + AudioSet + YT8M, Gradient Re-weighting (RW)=Video-Audio2022.11 | 61.06 | 89.66 | |
| VATT + GRPre-training Dataset=HT100M + AudioSet + YT8M, Cross-Modality Gradient Realignment (GR)=true2022.11 | 60.99 | 88.35 | |
| VATT + RW (VA)Pre-training Dataset=HT100M + AudioSet, Gradient Re-weighting (RW)=Video-Audio2022.11 | 59.55 | 88.02 | |
| VATT + RW (VA)Pre-training Dataset=HT100M, Gradient Re-weighting (RW)=Video-Audio2022.11 | 58.74 | 85.39 | |
| VATT + BothPre-training Dataset=HT100M, Cross-Modality Gradient Realignment (GR)=true, Gradient-based Curriculum Learning (CL)=true2022.11 | 58.24 | 87.37 | |
| VATT + RW (VT)Pre-training Dataset=HT100M, Gradient Re-weighting (RW)=Video-Text2022.11 | 58.23 | 86.8 | |
| VATTPre-training Dataset=HT100M2022.11 | 57.36 | 86.07 | |
| VATT + CLPre-training Dataset=HT100M, Gradient-based Curriculum Learning (CL)=true2022.11 | 56.41 | 86.06 | |
| VATT + GRPre-training Dataset=HT100M, Cross-Modality Gradient Realignment (GR)=true2022.11 | 54.38 | 83.64 | |
| SiGLIP 2Encoder=ViT-g/16, Protocol=Linear probing2026.02 | 47.83 | 73.87 |