# PO-GUISE+: Pose and object guided transformer token selection for efficient driver action recognition

Source: [PO-GUISE+: Pose and object guided transformer token selection for efficient driver action recognition](<https://www.sota2.com/research/paper/68fa74311e209131cea60544>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-04T05:00:05.078Z

Authors: Ricardo Pizarro, Roberto Valle, Rafael Barea, Jose M. Buenaposada, Luis Baumela, Luis Miguel Bergasa

Published: 2024-07-18T17:53:51.000Z

## Abstract

We address the task of identifying distracted driving by analyzing in-car videos using efficient transformers. Although transformer models have achieved outstanding performance in human action recognition tasks, their high computational costs limit their application onboard a vehicle. We introduce POGUISE+, a multi-task video transformer that, given an input clip, predicts the distracted driving action, the driver's pose, and the interacting object. Our enhanced features for token selection are specifically adapted to driver actions by leveraging information about object interaction and the driver's pose. With POGUISE+, we significantly reduce the model's computational demands while maintaining or improving baseline accuracy across various computational budgets. Additionally, to evaluate our model's performance in real-world scenarios, we have developed benchmarks on a Jetson computing platform, demonstrating its effectiveness across different configurations and computational budgets. Our model outperforms current state-of-the-art results on the Drive&Act, 100-Driver, and 3MDAD datasets, while having superior efficiency compared to existing video transformer-based methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Action Recognition on Drive&Act front-top NIR camera (test)](<https://www.sota2.com/research/sota/action-recognition-on-drive-and-act-front-top-nir-camera-test.md>): Action Recognition; Drive&Act front-top NIR camera (test); Macro Accuracy: 71.52.

- [Driver Action Recognition on 3MDAD averaged 5-fold (test)](<https://www.sota2.com/research/sota/driver-action-recognition-on-3mdad-averaged-5-fold-test.md>): Driver Action Recognition; 3MDAD averaged 5-fold (test); Accuracy: 93.42.

- [Driver Action Recognition on 100-Driver (test)](<https://www.sota2.com/research/sota/driver-action-recognition-on-100-driver-test.md>): Driver Action Recognition; 100-Driver (test); Accuracy: 93.54.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2407.13750>)

[PDF](<https://arxiv.org/pdf/2407.13750.pdf>)