# Fusing Hand and Body Skeletons for Human Action Recognition in Assembly

Source: [Fusing Hand and Body Skeletons for Human Action Recognition in Assembly](<https://www.sota2.com/research/paper/68fa701e1e209131cea49263>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:34:16.167Z

Authors: Dustin Aganian, Mona K\\"ohler, Benedict Stephan, Markus Eisenbach, Horst-Michael Gross

Published: 2023-07-18T13:18:52.000Z

## Abstract

As collaborative robots (cobots) continue to gain popularity in industrial manufacturing, effective human-robot collaboration becomes crucial. Cobots should be able to recognize human actions to assist with assembly tasks and act autonomously. To achieve this, skeleton-based approaches are often used due to their ability to generalize across various people and environments. Although body skeleton approaches are widely used for action recognition, they may not be accurate enough for assembly actions where the worker's fingers and hands play a significant role. To address this limitation, we propose a method in which less detailed body skeletons are combined with highly detailed hand skeletons. We investigate CNNs and transformers, the latter of which are particularly adept at extracting and combining important information from both skeleton types using attention. This paper demonstrates the effectiveness of our proposed approach in enhancing action recognition in assembly scenarios.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Bargaining on AmazonHistoryPrice MI 1.0 (test)](<https://www.sota2.com/research/sota/bargaining-on-amazonhistoryprice-mi-1-0-test.md>): Bargaining; AmazonHistoryPrice MI 1.0 (test); Count: 682.

- [Bargaining on AmazonHistoryPrice ALL 1.0 (test)](<https://www.sota2.com/research/sota/bargaining-on-amazonhistoryprice-all-1-0-test.md>): Bargaining; AmazonHistoryPrice ALL 1.0 (test); Count: 837.

- [Bargaining on AmazonHistoryPrice CI 1.0 (test)](<https://www.sota2.com/research/sota/bargaining-on-amazonhistoryprice-ci-1-0-test.md>): Bargaining; AmazonHistoryPrice CI 1.0 (test); Count: 40.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.09238>)

[PDF](<https://arxiv.org/pdf/2307.09238.pdf>)