# Long-term Leap Attention, Short-term Periodic Shift for Video Classification

Source: [Long-term Leap Attention, Short-term Periodic Shift for Video Classification](<https://www.sota2.com/research/paper/68fa6d141e209131cea3783d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:24:36.116Z

Authors: Hao Zhang, Lechao Cheng, Yanbin Hao, Chong-Wah Ngo

Published: 2022-07-12T13:30:15.000Z

## Abstract

Video transformer naturally incurs a heavier computation burden than a static vision transformer, as the former processes $T$ times longer sequence than the latter under the current attention of quadratic complexity $(T^2N^2)$. The existing works treat the temporal axis as a simple extension of spatial axes, focusing on shortening the spatio-temporal sequence by either generic pooling or local windowing without utilizing temporal redundancy.   However, videos naturally contain redundant information between neighboring frames; thereby, we could potentially suppress attention on visually similar frames in a dilated manner. Based on this hypothesis, we propose the LAPS, a long-term \`\`\\textbf{\\textit{Leap Attention}}'' (LA), short-term \`\`\\textbf{\\textit{Periodic Shift}}'' (\\textit{P}-Shift) module for video transformers, with $(2TN^2)$ complexity. Specifically, the \`\`LA'' groups long-term frames into pairs, then refactors each discrete pair via attention. The \`\`\\textit{P}-Shift'' exchanges features between temporal neighbors to confront the loss of short-term dynamics. By replacing a vanilla 2D attention with the LAPS, we could adapt a static transformer into a video one, with zero extra parameters and neglectable computation overhead ($\\sim$2.6\\%). Experiments on the standard Kinetics-400 benchmark demonstrate that our LAPS transformer could achieve competitive performances in terms of accuracy, FLOPs, and Params among CNN and transformer SOTAs. We open-source our project in \\sloppy \\href{https://github.com/VideoNetworks/LAPS-transformer}{\\textit{\\color{magenta}{https://github.com/VideoNetworks/LAPS-transformer}}} .

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.05526>)

[PDF](<https://arxiv.org/pdf/2207.05526.pdf>)
