# Spatiotemporal CNN for Video Object Segmentation

Source: [Spatiotemporal CNN for Video Object Segmentation](<https://www.sota2.com/research/paper/68fa658a1e209131cea0af06>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:08:58.379Z

Authors: Kai Xu, Longyin Wen, Guorong Li, Liefeng Bo, Qingming Huang

Published: 2019-04-04T05:53:15.000Z

## Abstract

In this paper, we present a unified, end-to-end trainable spatiotemporal CNN model for VOS, which consists of two branches, i.e., the temporal coherence branch and the spatial segmentation branch. Specifically, the temporal coherence branch pretrained in an adversarial fashion from unlabeled video data, is designed to capture the dynamic appearance and motion cues of video sequences to guide object segmentation. The spatial segmentation branch focuses on segmenting objects accurately based on the learned appearance and motion cues. To obtain accurate segmentation results, we design a coarse-to-fine process to sequentially apply a designed attention module on multi-scale feature maps, and concatenate them to produce the final prediction. In this way, the spatial segmentation branch is enforced to gradually concentrate on object regions. These two branches are jointly fine-tuned on video segmentation sequences in an end-to-end manner. Several experiments are carried out on three challenging datasets (i.e., DAVIS-2016, DAVIS-2017 and Youtube-Object) to show that our method achieves favorable performance against the state-of-the-arts. Code is available at https://github.com/longyin880815/STCNN.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Object Segmentation on YouTube-Objects](<https://www.sota2.com/research/sota/video-object-segmentation-on-youtube-objects.md>): Video Object Segmentation; YouTube-Objects; mIoU: 79.6.

- [Video Object Segmentation on DAVIS 2016 (val)](<https://www.sota2.com/research/sota/video-object-segmentation-on-davis-2016-val.md>): Video Object Segmentation; DAVIS 2016 (val); J Mean: 83.8.

- [Video Object Segmentation on DAVIS 2017 (val)](<https://www.sota2.com/research/sota/video-object-segmentation-on-davis-2017-val.md>): Video Object Segmentation; DAVIS 2017 (val); J mean: 58.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1904.02363>)

[PDF](<https://arxiv.org/pdf/1904.02363.pdf>)

[Implementation](<https://github.com/longyin880815/STCNN>)
