# Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition

Source: [Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition](<https://www.sota2.com/research/paper/68fa6cfe1e209131cea372ad>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:16:31.054Z

Authors: Yansong Tang, Xingyu Liu, Xumin Yu, Danyang Zhang, Jiwen Lu, Jie Zhou

Published: 2022-07-17T07:05:39.000Z

## Abstract

Rapid progress and superior performance have been achieved for skeleton-based action recognition recently. In this article, we investigate this problem under a cross-dataset setting, which is a new, pragmatic, and challenging task in real-world scenarios. Following the unsupervised domain adaptation (UDA) paradigm, the action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. Different from the conventional adversarial learning-based approaches for UDA, we utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. Our inspiration is drawn from Cubism, an art genre from the early 20th century, which breaks and reassembles the objects to convey a greater context. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks to explore the temporal and spatial dependency of a skeleton-based action and improve the generalization ability of the model. We conduct experiments on six datasets for skeleton-based action recognition, including three large-scale datasets (NTU RGB+D, PKU-MMD, and Kinetics) where new cross-dataset settings and benchmarks are established. Extensive results demonstrate that our method outperforms state-of-the-art approaches. The source codes of our model and all the compared methods are available at https://github.com/shanice-l/st-cubism.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Skeleton Action Recognition on NTU, PKU, and ETRI Cross-dataset](<https://www.sota2.com/research/sota/skeleton-action-recognition-on-ntu-pku-and-etri-cross-dataset.md>): Skeleton Action Recognition; NTU, PKU, and ETRI Cross-dataset; N -\> E Accuracy: 59.1.

- [Action Recognition on NTU-RGB+D 51 (N51) to PKU-MMD 51 (P51) (cross-dataset)](<https://www.sota2.com/research/sota/action-recognition-on-ntu-rgb-plus-d-51-n51-to-pku-mmd-51-p51-cross-dataset.md>): Action Recognition; NTU-RGB+D 51 (N51) to PKU-MMD 51 (P51) (cross-dataset); Accuracy: 70.5.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.08095>)

[PDF](<https://arxiv.org/pdf/2207.08095.pdf>)
