Action Recognition on Toyota Smarthome CS
73.1Accuracyπ-ViT + 3D Poses
Evaluation results
| Method | Links | ||
|---|---|---|---|
| π-ViT + 3D PosesPose=Yes, RGB=Yes2023.11 | 73.1 | — | |
| π-ViT + 3D Poses+Skeleton Modality=true, RGB Modality=true, 10 clips per video=true2024.08 | 73.1 | — | |
| π-ViTPose=Only during training, RGB=Yes2023.11 | 72.9 | — | |
| π-ViTSkeleton Modality=false, RGB Modality=true, 10 clips per video=true, Skeleton used in training but not in inference=true2024.08 | 72.9 | — | |
| TimeSformer + 2D-SIMPose=Only during training, RGB=Yes2023.11 | 72.5 | — | |
| EPAM-NetSkeleton Modality=true, RGB Modality=true2024.08 | 71.7 | — | |
| TimeSformer + 3D-SIMPose=Only during training, RGB=Yes2023.11 | 71.4 | — | |
| VPN++ + 3D PosesPose=Yes, RGB=Yes2023.11 | 71 | — | |
| VPN+++ 3D PosesSkeleton Modality=true, RGB Modality=true2024.08 | 71 | — | |
| Ours - AM/2(1,12), LSTMRGB=true, Skeleton=false, Pretrain=IN-21K, #F=82024.11 | 70.2 | — | |
| MMNetRGB=true, Skeleton=false, Pretrain=Scratch, #F=-2024.11 | 70.1 | — | |
| Video SwinPose=No, RGB=Yes2023.11 | 69.8 | — | |
| VPN++Pose=Only during training, RGB=Yes2023.11 | 69 | — | |
| VPN++RGB=true, Skeleton=true, Pretrain=Scratch, #F=64+pose2024.11 | 69 | — | |
| VPN++Skeleton Modality=false, RGB Modality=true, Skeleton used in training but not in inference=true2024.08 | 69 | — | |
| TimeSformerPose=No, RGB=Yes2023.11 | 68.4 | — | |
| LTNPose=No, RGB=Yes2023.11 | 65.9 | — | |
| MotionFormerPose=No, RGB=Yes2023.11 | 65.8 | — | |
| VPNPose=Yes, RGB=Yes2023.11 | 65.2 | — | |
| VPN (I3D)Skeleton Modality=true, RGB Modality=true, backbone=I3D2024.08 | 65.2 | — | |
| DG-STGCNModality=2-stream (joint+bone), Pre-training=NTU120-XSet2022.10 | 65.1 | — | |
| MMNet (ResNet18)Skeleton Modality=true, RGB Modality=true, backbone=ResNet182024.08 | 65.1 | — | |
| DG-STGCNModality=2-stream (joint+bone)2022.10 | 64.8 | — | |
| UNIKRGB=false, Pose=true, Pre-training=Posetics(Ours)2021.07 | 64.3 | — | |
| UNIK++* w. PRSModality=2-stream (joint+bone), Pose Input Quality=high-quality 2D Pose, Pre-training=Pre-trained on another dataset, w. PRS=true2022.10 | 64.3 | — | |
| ST-GCNRGB=false, Skeleton=true, Pretrain=Posetics, #F=-2024.11 | 64.3 | — | |
| UNIKRGB=false, Skeleton=true, Pretrain=K400, #F=642024.11 | 64.3 | — | |
| AssembleNet++RGB=true, Pose=false, Pre-training=Kinetics-4002021.07 | 63.6 | — | |
| AssembleNet++Pose=No, RGB=Yes2023.11 | 63.6 | — | |
| MS-G3DRGB=false, Skeleton=true, Pretrain=K400, #F=642024.11 | 63.6 | — | |
| AssembleNet++RGB=true, Skeleton=false, Pretrain=K400, #F=642024.11 | 63.6 | — | |
| UNIKRGB=false, Pose=true, Pre-training=Scratch2021.07 | 63.1 | — | |
| MS-G3D NetRGB=false, Pose=true, Pre-training=Scratch2021.07 | 61.1 | — | |
| 2s-AGCNRGB=false, Pose=true, Pre-training=Scratch2021.07 | 60.9 | — | |
| AGCN++ w. PRSModality=2-stream (joint+bone), Pose Input Quality=high-quality 2D Pose, w. PRS=true2022.10 | 60.9 | — | |
| 2s-AGCNPose=Yes, RGB=No2023.11 | 60.9 | — | |
| VPNRGB=true, Pose=false, Pre-training=Kinetics-4002021.07 | 60.8 | — | |
| VPNRGB=true, Skeleton=true, Pretrain=Scratch, #F=64+pose2024.11 | 60.8 | — | |
| HyperformerPose=Yes, RGB=No2023.11 | 57.5 | — | |
| AGCN++Modality=2-stream (joint+bone), Pose Input Quality=high-quality 2D Pose2022.10 | 57.1 | — | |
| TimeSformer + Fibottention (Modified Wythoff)Backbone=TimeSformer, Instantiation=Modified Wythoff2024.06 | 57.1 | — | |
| MS-AAGCN2022.10 | 56.5 | — | |
| TimeSformer + Fibottention (Wythoff)Backbone=TimeSformer, Instantiation=Wythoff2024.06 | 55.6 | — | |
| Separable STARGB=true, Pose=false, Pre-training=Kinetics-4002021.07 | 54.2 | — | |
| P-I3DPose=Yes, RGB=Yes2023.11 | 54.2 | — | |
| Separable STAPose=Yes, RGB=Yes2023.11 | 54.2 | — | |
| Separable STARGB=true, Skeleton=false, Pretrain=Scratch, #F=642024.11 | 54.2 | — | |
| ST-GCNRGB=false, Pose=true, Pre-training=Scratch2021.07 | 53.8 | — | |
| PoseC3DPose=Yes, RGB=Yes2023.11 | 53.8 | — | |
| TSMFSkeleton Modality=true, RGB Modality=true2024.08 | 53.8 | — | |
| I3DRGB=true, Pose=false, Pre-training=Kinetics-4002021.07 | 53.4 | — | |
| 2s-AGCNRGB=false, Skeleton=true, Pretrain=K400, #F=642024.11 | 53.4 | — | |
| I3DRGB=true, Skeleton=false, Pretrain=K400, #F=642024.11 | 53.4 | — | |
| TimeSformerBackbone=TimeSformer2024.06 | 52.2 | — | |
| TimeSformer + BigBirdBackbone=TimeSformer, Attention Mechanism=BigBird2024.06 | 51.4 | — | |
| PoseC3DPose=Yes, RGB=No2023.11 | 50.6 | — | |
| LSTM2022.10 | 42.5 | — | |
| LSTMRGB=false, Pose=true, Pre-training=Scratch2021.07 | 12.5 | — | |
| 2s-AGCNModality - 2D Ske=false, Modality - 3D Ske=true, Modality - RGB=false, Modeling Paradigm=Unimodal2025.12 | — | 60.9 | |
| 2s-AGCNPose=Yes, RGB=No2026.03 | — | 60.9 | |
| AssembleNet++Pose=No, RGB=Yes2026.03 | — | 63.6 | |
| HyperformerModality - 2D Ske=false, Modality - 3D Ske=true, Modality - RGB=false, Modeling Paradigm=Unimodal2025.12 | — | 57.5 | |
| I3DPose=No, RGB=Yes2026.03 | — | 53.4 | |
| LTNPose=No, RGB=Yes2026.03 | — | 65.9 | |
| MMNetPose=Yes, RGB=Yes2026.03 | — | 70.1 | |
| MotionFormerModality - 2D Ske=false, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Unimodal2025.12 | — | 65.8 | |
| MS-G3D NetPose=Yes, RGB=No2026.03 | — | 66.7 | |
| P-I3DPose=Yes, RGB=Yes2026.03 | — | 54.2 | |
| PAN (Even)Modality - 2D Ske=false, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Unimodal, Sampling Strategy=zero padding2025.12 | — | 70.5 | |
| PAN (Even)Modality - 2D Ske=false, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Unimodal, Sampling Strategy=token replication2025.12 | — | 70.8 | |
| PAN (Guided)Modality - 2D Ske=true, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling)2025.12 | — | 69.9 | |
| PAN-EnsembleModality - 2D Ske=false, Modality - 3D Ske=true, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling)2025.12 | — | 73 | |
| PAN-UnifiedModality - 2D Ske=false, Modality - 3D Ske=true, Modality - RGB=true, Modeling Paradigm=Multimodal (unified modeling)2025.12 | — | 71 | |
| PoseC3DModality - 2D Ske=re-estimated data, Modality - 3D Ske=false, Modality - RGB=false, Modeling Paradigm=Unimodal2025.12 | — | 50.6 | |
| PoseC3DModality - 2D Ske=re-estimated data, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling)2025.12 | — | 53.8 | |
| Proposed Multi-modal ArchitecturePose=Yes, RGB=Yes2026.03 | — | 70.1 | |
| Separable STAPose=Yes, RGB=Yes2026.03 | — | 54.2 | |
| ST-GCNPose=Yes, RGB=No2026.03 | — | 53.8 | |
| SV-data2vecPose=Training Only, RGB=Yes2026.03 | — | 72.9 | |
| TimeSformerModality - 2D Ske=false, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Unimodal2025.12 | — | 68.4 | |
| TimeSformerPre-train=Kinetics400, Resolution=224x224, Frames=82026.06 | — | 67.5 | |
| TimeSformer + Diff. Attn.Pre-train=Kinetics400, Resolution=224x224, Frames=82026.06 | — | 66.6 | |
| TimeSformer + DnAPre-train=Kinetics400, Resolution=224x224, Frames=82026.06 | — | 68.8 | |
| UNIKPose=Yes, RGB=No2026.03 | — | 63.1 | |
| ViAPose=Yes, RGB=No2026.03 | — | 64 | |
| Video SwinModality - 2D Ske=false, Modality - 3D Ske=false, Modality - RGB=true, Modeling Paradigm=Unimodal2025.12 | — | 69.8 | |
| VPNPose=Yes, RGB=Yes2026.03 | — | 65.2 | |
| VPN++Modality - 2D Ske=false, Modality - 3D Ske=training only, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling)2025.12 | — | 69 | |
| VPN++Modality - 2D Ske=false, Modality - 3D Ske=true, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling), Fusion=late fusion2025.12 | — | 71 | |
| VPN++Pose=Training Only, RGB=Yes2026.03 | — | 69 | |
| VPN++ + 3D PosesPose=Yes, RGB=Yes2026.03 | — | 71 | |
| π-ViTModality - 2D Ske=training only, Modality - 3D Ske=training only, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling)2025.12 | — | 72.9 | |
| π-ViTModality - 2D Ske=training only, Modality - 3D Ske=true, Modality - RGB=true, Modeling Paradigm=Multimodal (separate modeling), Fusion=late fusion2025.12 | — | 73.1 | |
| π-ViTPose=Training Only, RGB=Yes2026.03 | — | 72.9 | |
| π-ViT + 3D PosesPose=Yes, RGB=Yes2026.03 | — | 73.1 |