# Exploring Context with Deep Structured models for Semantic Segmentation Source: [Exploring Context with Deep Structured models for Semantic Segmentation]() Collection index: [llms.txt]() Data updated: 2025-10-25T20:37:15.413Z Authors: Guosheng Lin, Chunhua Shen, Anton van den Hengel, Ian Reid Published: 2016-03-10T08:34:19.000Z ## Abstract State-of-the-art semantic image segmentation methods are mostly based on training deep convolutional neural networks (CNNs). In this work, we proffer to improve semantic segmentation with the use of contextual information. In particular, we explore \`patch-patch' context and \`patch-background' context in deep CNNs. We formulate deep structured models by combining CNNs and Conditional Random Fields (CRFs) for learning the patch-patch context between image regions. Specifically, we formulate CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied in order to avoid repeated expensive CRF inference during the course of back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image inputs and sliding pyramid pooling is very effective for improving performance. We perform comprehensive evaluation of the proposed method. We achieve new state-of-the-art performance on a number of challenging semantic segmentation datasets including $NYUDv2$, $PASCAL$-$VOC2012$, $Cityscapes$, $PASCAL$-$Context$, $SUN$-$RGBD$, $SIFT$-$flow$, and $KITTI$ datasets. Particularly, we report an intersection-over-union score of $77.8$ on the $PASCAL$-$VOC2012$ dataset. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Semantic Segmentation on SUN RGB-D (test)](): Semantic segmentation; SUN RGB-D (test); mIoU: 42.3. - [Semantic Segmentation on PASCAL Context (val)](): Semantic segmentation; PASCAL Context (val); mIoU: 43.3. - [Semantic Segmentation on NYU v2 (test)](): Semantic segmentation; NYU v2 (test); mIoU: 40.6. - [Semantic Segmentation on PASCAL VOC 2012 (test)](): Semantic segmentation; PASCAL VOC 2012 (test); mIoU: 75.3. - [Semantic Segmentation on Cityscapes (test)](): Semantic segmentation; Cityscapes (test); mIoU: not recorded. ## Original sources and implementation [Original paper]() [PDF]()