# Skip-Convolutions for Efficient Video Processing

Source: [Skip-Convolutions for Efficient Video Processing](<https://www.sota2.com/research/paper/68fa69cc1e209131cea241ac>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:08:38.583Z

Authors: Amirhossein Habibian, Davide Abati, Taco S. Cohen, Babak Ehteshami Bejnordi

Published: 2021-04-23T09:10:39.000Z

## Abstract

We propose Skip-Convolutions to leverage the large amount of redundancies in video streams and save computations. Each video is represented as a series of changes across frames and network activations, denoted as residuals. We reformulate standard convolution to be efficiently computed on residual frames: each layer is coupled with a binary gate deciding whether a residual is important to the model prediction,~\\eg foreground regions, or it can be safely skipped, e.g. background regions. These gates can either be implemented as an efficient network trained jointly with convolution kernels, or can simply skip the residuals based on their magnitude. Gating functions can also incorporate block-wise sparsity structures, as required for efficient implementation on hardware platforms. By replacing all convolutions with Skip-Convolutions in two state-of-the-art architectures, namely EfficientDet and HRNet, we reduce their computational cost consistently by a factor of 3~4x for two different tasks, without any accuracy drop. Extensive comparisons with existing model compression, as well as image and video efficiency methods demonstrate that Skip-Convolutions set a new state-of-the-art by effectively exploiting the temporal redundancies in videos.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Online Pose Estimation on Sub-JHMDB (test)](<https://www.sota2.com/research/sota/online-pose-estimation-on-sub-jhmdb-test.md>): Online pose estimation; Sub-JHMDB (test); Head Acc: 98.7.

- [Semantic Segmentation on Cityscapes (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-cityscapes-val.md>): Semantic segmentation; Cityscapes (val); mIoU: 75.5.

- [Video Object Detection on ImageNet VID (val)](<https://www.sota2.com/research/sota/video-object-detection-on-imagenet-vid-val.md>): Video Object Detection; ImageNet VID (val); mAP (%): 62.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2104.11487>)

[PDF](<https://arxiv.org/pdf/2104.11487.pdf>)