# VideoLT: Large-scale Long-tailed Video Recognition

Source: [VideoLT: Large-scale Long-tailed Video Recognition](<https://www.sota2.com/research/paper/68fa6a8e1e209131cea28c4b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:09:35.928Z

Authors: Xing Zhang, Zuxuan Wu, Zejia Weng, Huazhu Fu, Jingjing Chen, Yu-Gang Jiang, Larry Davis

Published: 2021-05-06T13:47:44.000Z

## Abstract

Label distributions in real-world are oftentimes long-tailed and imbalanced, resulting in biased models towards dominant labels. While long-tailed recognition has been extensively studied for image classification tasks, limited effort has been made for video domain. In this paper, we introduce VideoLT, a large-scale long-tailed video recognition dataset, as a step toward real-world video recognition. Our VideoLT contains 256,218 untrimmed videos, annotated into 1,004 classes with a long-tailed distribution. Through extensive studies, we demonstrate that state-of-the-art methods used for long-tailed image recognition do not perform well in the video domain due to the additional temporal dimension in video data. This motivates us to propose FrameStack, a simple yet effective method for long-tailed video recognition task. In particular, FrameStack performs sampling at the frame-level in order to balance class distributions, and the sampling ratio is dynamically determined using knowledge derived from the network during training. Experimental results demonstrate that FrameStack can improve classification performance without sacrificing overall accuracy. Code and dataset are available at: https://github.com/17Skye17/VideoLT.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Action Recognition on EPIC-KITCHENS-100 Verbs (val)](<https://www.sota2.com/research/sota/action-recognition-on-epic-kitchens-100-verbs-val.md>): Action Recognition; EPIC-KITCHENS-100 Verbs (val); Few Performance: 23.

- [Action Recognition on SSv2-LT](<https://www.sota2.com/research/sota/action-recognition-on-ssv2-lt.md>): Action Recognition; SSv2-LT; Accuracy (Few): 15.5.

- [Action Recognition on VideoLT-LT](<https://www.sota2.com/research/sota/action-recognition-on-videolt-lt.md>): Action Recognition; VideoLT-LT; Few-Shot Acc: 18.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2105.02668>)

[PDF](<https://arxiv.org/pdf/2105.02668.pdf>)
