# MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

Source: [MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features](<https://www.sota2.com/research/paper/68fa63eb1e209131cea01740>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-24T01:10:32.202Z

Authors: Liang-Chieh Chen, Alexander Hermans, George Papandreou, Florian Schroff, Peng Wang, Hartwig Adam

Published: 2017-12-13T16:09:55.000Z

## Abstract

In this work, we tackle the problem of instance segmentation, the task of simultaneously solving object detection and semantic segmentation. Towards this goal, we present a model, called MaskLab, which produces three outputs: box detection, semantic segmentation, and direction prediction. Building on top of the Faster-RCNN object detector, the predicted boxes provide accurate localization of object instances. Within each region of interest, MaskLab performs foreground/background segmentation by combining semantic and direction prediction. Semantic segmentation assists the model in distinguishing between objects of different semantic classes including background, while the direction prediction, estimating each pixel's direction towards its corresponding center, allows separating instances of the same semantic class. Moreover, we explore the effect of incorporating recent successful methods from both segmentation and detection (i.e. atrous convolution and hypercolumn). Our proposed model is evaluated on the COCO instance segmentation benchmark and shows comparable performance with other state-of-art models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Instance Segmentation on COCO 2017 (test-dev)](<https://www.sota2.com/research/sota/instance-segmentation-on-coco-2017-test-dev.md>): Instance Segmentation; COCO 2017 (test-dev); AP (Overall): 38.1.

- [Instance Segmentation on COCO (test-dev)](<https://www.sota2.com/research/sota/instance-segmentation-on-coco-test-dev.md>): Instance Segmentation; COCO (test-dev); APM: 39.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1712.04837>)

[PDF](<https://arxiv.org/pdf/1712.04837.pdf>)