# MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentation

Source: [MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentation](<https://www.sota2.com/research/paper/68fa71621e209131cea5108c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:58:02.282Z

Authors: Abdullah Rashwan, Jiageng Zhang, Ali Taalimi, Fan Yang, Xingyi Zhou, Chaochao Yan, Liang-Chieh Chen, Yeqing Li

Published: 2023-12-11T00:52:26.000Z

## Abstract

In recent years, transformer-based models have dominated panoptic segmentation, thanks to their strong modeling capabilities and their unified representation for both semantic and instance classes as global binary masks. In this paper, we revisit pure convolution model and propose a novel panoptic architecture named MaskConver. MaskConver proposes to fully unify things and stuff representation by predicting their centers. To that extent, it creates a lightweight class embedding module that can break the ties when multiple centers co-exist in the same location. Furthermore, our study shows that the decoder design is critical in ensuring that the model has sufficient context for accurate detection and segmentation. We introduce a powerful ConvNeXt-UNet decoder that closes the performance gap between convolution- and transformerbased models. With ResNet50 backbone, our MaskConver achieves 53.6% PQ on the COCO panoptic val set, outperforming the modern convolution-based model, Panoptic FCN, by 9.3% as well as transformer-based models such as Mask2Former (+1.7% PQ) and kMaX-DeepLab (+0.6% PQ). Additionally, MaskConver with a MobileNet backbone reaches 37.2% PQ, improving over Panoptic-DeepLab by +6.4% under the same FLOPs/latency constraints. A further optimized version of MaskConver achieves 29.7% PQ, while running in real-time on mobile devices. The code and model weights will be publicly available

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Panoptic Segmentation on COCO (val)](<https://www.sota2.com/research/sota/panoptic-segmentation-on-coco-val.md>): Panoptic Segmentation; COCO (val); PQ: 53.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2312.06052>)

[PDF](<https://arxiv.org/pdf/2312.06052.pdf>)

[Implementation](<https://github.com/tensorflow/models/tree/master/official/projects/maskconver>)
