# PMI Sampler: Patch Similarity Guided Frame Selection for Aerial Action Recognition

Source: [PMI Sampler: Patch Similarity Guided Frame Selection for Aerial Action Recognition](<https://www.sota2.com/research/paper/68fa713f1e209131cea50136>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:48:32.139Z

Authors: Ruiqi Xian, Xijun Wang, Divya Kothandaraman, Dinesh Manocha

Published: 2023-04-14T00:01:11.000Z

## Abstract

We present a new algorithm for selection of informative frames in video action recognition. Our approach is designed for aerial videos captured using a moving camera where human actors occupy a small spatial resolution of video frames. Our algorithm utilizes the motion bias within aerial videos, which enables the selection of motion-salient frames. We introduce the concept of patch mutual information (PMI) score to quantify the motion bias between adjacent frames, by measuring the similarity of patches. We use this score to assess the amount of discriminative motion information contained in one frame relative to another. We present an adaptive frame selection strategy using shifted leaky ReLu and cumulative distribution function, which ensures that the sampled frames comprehensively cover all the essential segments with high motion salience. Our approach can be integrated with any action recognition model to enhance its accuracy. In practice, our method achieves a relative improvement of 2.2 - 13.8% in top-1 accuracy on UAV-Human, 6.8% on NEC Drone, and 9.0% on Diving48 datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Action Recognition on UAV Human (test)](<https://www.sota2.com/research/sota/action-recognition-on-uav-human-test.md>): Action Recognition; UAV Human (test); Top-1 Accuracy: 55.

- [Action Recognition on UAV-Human](<https://www.sota2.com/research/sota/action-recognition-on-uav-human.md>): Action Recognition; UAV-Human; Top-1 Acc: 55.

- [Action Recognition on Diving-48 (val)](<https://www.sota2.com/research/sota/action-recognition-on-diving-48-val.md>): Action Recognition; Diving-48 (val); Accuracy: 81.3.

- [Action Recognition on HMDB51 (avg 3 splits)](<https://www.sota2.com/research/sota/action-recognition-on-hmdb51-avg-3-splits.md>): Action Recognition; HMDB51 (avg 3 splits); Accuracy: 65.5.

- [Action Recognition on NEC Drones](<https://www.sota2.com/research/sota/action-recognition-on-nec-drones.md>): Action Recognition; NEC Drones; Top-1 Accuracy: 62.5.

- [Action Recognition on Diving-48](<https://www.sota2.com/research/sota/action-recognition-on-diving-48.md>): Action Recognition; Diving-48; Top-1 Acc: 81.3.

- [Action Recognition on Something-Something v2 (test)](<https://www.sota2.com/research/sota/action-recognition-on-something-something-v2-test.md>): Action Recognition; Something-Something v2 (test); Top-1 Acc: 60.3.

- [Action Recognition on UCF101 (mean of 3 splits)](<https://www.sota2.com/research/sota/action-recognition-on-ucf101-mean-of-3-splits.md>): Action Recognition; UCF101 (mean of 3 splits); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2304.06866>)

[PDF](<https://arxiv.org/pdf/2304.06866.pdf>)

[Implementation](<https://github.com/Ricky-Xian/PMI-Sampler>)
