# DynamoNet: Dynamic Action and Motion Network

Source: [DynamoNet: Dynamic Action and Motion Network](<https://www.sota2.com/research/paper/68fa65ab1e209131cea0bce3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:32:37.817Z

Authors: Ali Diba, Vivek Sharma, Luc Van Gool, Rainer Stiefelhagen

Published: 2019-04-25T15:27:12.000Z

## Abstract

In this paper, we are interested in self-supervised learning the motion cues in videos using dynamic motion filters for a better motion representation to finally boost human action recognition in particular. Thus far, the vision community has focused on spatio-temporal approaches using standard filters, rather we here propose dynamic filters that adaptively learn the video-specific internal motion representation by predicting the short-term future frames. We name this new motion representation, as dynamic motion representation (DMR) and is embedded inside of 3D convolutional network as a new layer, which captures the visual appearance and motion dynamics throughout entire video clip via end-to-end network learning. Simultaneously, we utilize these motion representation to enrich video classification. We have designed the frame prediction task as an auxiliary task to empower the classification problem. With these overall objectives, to this end, we introduce a novel unified spatio-temporal 3D-CNN architecture (DynamoNet) that jointly optimizes the video classification and learning motion representation by predicting future frames as a multi-task learning problem. We conduct experiments on challenging human action datasets: Kinetics 400, UCF101, HMDB51. The experiments using the proposed DynamoNet show promising results on all the datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Action Classification on HMDB51 (split1)](<https://www.sota2.com/research/sota/action-classification-on-hmdb51-split1.md>): Action Classification; HMDB51 (split1); Accuracy: 59.9.

- [Action Recognition on UCF101 (Split 1)](<https://www.sota2.com/research/sota/action-recognition-on-ucf101-split-1.md>): Action Recognition; UCF101 (Split 1); Top-1 Acc: 88.1.

- [Video Classification on Kinetics 400 (val)](<https://www.sota2.com/research/sota/video-classification-on-kinetics-400-val.md>): Video Classification; Kinetics 400 (val); Top-1 Acc: 77.9.

- [Action Recognition on Kinetics-400](<https://www.sota2.com/research/sota/action-recognition-on-kinetics-400.md>): Action Recognition; Kinetics-400; Top-1 Acc: 77.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1904.11407>)

[PDF](<https://arxiv.org/pdf/1904.11407.pdf>)
