# Weakly-Supervised Semantic Segmentation using Motion Cues Source: [Weakly-Supervised Semantic Segmentation using Motion Cues]() Collection index: [llms.txt]() Data updated: 2025-10-25T20:38:15.653Z Authors: Pavel Tokmakov, Karteek Alahari, Cordelia Schmid Published: 2016-03-23T14:01:03.000Z ## Abstract Fully convolutional neural networks (FCNNs) trained on a large number of images with strong pixel-level annotations have become the new state of the art for the semantic segmentation task. While there have been recent attempts to learn FCNNs from image-level weak annotations, they need additional constraints, such as the size of an object, to obtain reasonable performance. To address this issue, we present motion-CNN (M-CNN), a novel FCNN framework which incorporates motion cues and is learned from video-level weak annotations. Our learning scheme to train the network uses motion segments as soft constraints, thereby handling noisy motion information. When trained on weakly-annotated videos, our method outperforms the state-of-the-art EM-Adapt approach on the PASCAL VOC 2012 image segmentation benchmark. We also demonstrate that the performance of M-CNN learned with 150 weak video annotations is on par with state-of-the-art weakly-supervised methods trained with thousands of images. Finally, M-CNN substantially outperforms recent approaches in a related task of video co-localization on the YouTube-Objects dataset. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Weakly Supervised Semantic Segmentation on PASCAL VOC 2012 (test)](): Weakly supervised semantic segmentation; PASCAL VOC 2012 (test); mIoU: 39.8. - [Weakly Supervised Semantic Segmentation on PASCAL VOC 2012 (val)](): Weakly supervised semantic segmentation; PASCAL VOC 2012 (val); mIoU: 38.1. - [Semantic Segmentation on PASCAL VOC 2012 (test)](): Semantic segmentation; PASCAL VOC 2012 (test); mIoU: 39.8. - [Semantic Segmentation on PASCAL VOC 2012 (val)](): Semantic segmentation; PASCAL VOC 2012 (val); Mean IoU: 38.1. ## Original sources and implementation [Original paper]() [PDF]()