# Spatio-Temporal Learnable Proposals for End-to-End Video Object Detection

Source: [Spatio-Temporal Learnable Proposals for End-to-End Video Object Detection](<https://www.sota2.com/research/paper/68fa6da41e209131cea3b157>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:43:16.594Z

Authors: Khurram Azeem Hashmi, Didier Stricker, Muhammamd Zeshan Afzal

Published: 2022-10-05T16:17:55.000Z

## Abstract

This paper presents the novel idea of generating object proposals by leveraging temporal information for video object detection. The feature aggregation in modern region-based video object detectors heavily relies on learned proposals generated from a single-frame RPN. This imminently introduces additional components like NMS and produces unreliable proposals on low-quality frames. To tackle these restrictions, we present SparseVOD, a novel video object detection pipeline that employs Sparse R-CNN to exploit temporal information. In particular, we introduce two modules in the dynamic head of Sparse R-CNN. First, the Temporal Feature Extraction module based on the Temporal RoI Align operation is added to extract the RoI proposal features. Second, motivated by sequence-level semantic aggregation, we incorporate the attention-guided Semantic Proposal Feature Aggregation module to enhance object feature representation before detection. The proposed SparseVOD effectively alleviates the overhead of complicated post-processing methods and makes the overall pipeline end-to-end trainable. Extensive experiments show that our method significantly improves the single-frame Sparse RCNN by 8%-9% in mAP. Furthermore, besides achieving state-of-the-art 80.3% mAP on the ImageNet VID dataset with ResNet-50 backbone, our SparseVOD outperforms existing proposal-based methods by a significant margin on increasing IoU thresholds (IoU \> 0.5).

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Object Detection on ImageNet VID (val)](<https://www.sota2.com/research/sota/video-object-detection-on-imagenet-vid-val.md>): Video Object Detection; ImageNet VID (val); mAP (%): 81.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2210.02368>)

[PDF](<https://arxiv.org/pdf/2210.02368.pdf>)