Panoptic Segmentation on COCO (val)
59.5PQOpenSeeD (L)
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OpenSeeD (L)Type=Open-vocabulary, Backbone=Large, Evaluation Protocol=Without fine-tuning2023.03 | 59.5 | — | — | — | — | — | — | — | — | — | — | — | |
| MaskDINO (L)Type=Closed-set, Backbone=Large2023.03 | 58.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former + DejaVuBackbone=Swin-L2023.03 | 58 | — | — | 48.3 | — | — | 64.4 | — | — | — | — | — | |
| OneFormer (L)Type=Closed-set, Backbone=Large2023.03 | 57.9 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former (L)Type=Closed-set, Backbone=Large2023.03 | 57.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2FormerBackbone=Swin-L, Seg. Data=C2024.03 | 57.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2FormerBackbone=Swin-L2023.03 | 57.6 | — | — | 47.5 | — | — | 64.2 | — | — | — | — | — | |
| X-Decoder (L)Type=Open-vocabulary, Backbone=Large, Evaluation Protocol=Fine-tuned2023.03 | 56.9 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former (B)Type=Closed-set, Backbone=Base2023.03 | 56.4 | — | — | — | — | — | — | — | — | — | — | — | |
| X-DecoderBackbone=DaViT-B, Seg. Data=C, L, RC2024.03 | 56.2 | — | — | — | — | — | — | — | — | — | — | — | |
| SEEMBackbone=DaViT-B, Seg. Data=C, L, RC2024.03 | 56.1 | — | — | — | — | — | — | — | — | — | — | — | |
| PSALMBackbone=Swin-B, Seg. Data=C, RC, CI2024.03 | 55.9 | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic SegFormerBackbone=Swin-L†, Epochs=24, #P (M)=221.4, #F (G)=8162021.09 | 55.8 | — | — | 61.7 | — | — | 46.9 | — | — | — | — | — | |
| Panoptic SegFormerBackbone=PVTv2-B5†, Epochs=24, #P (M)=104.9, #F (G)=3492021.09 | 55.4 | — | — | 61.2 | — | — | 46.6 | — | — | — | — | — | |
| Pano/SegFormer (B)Type=Closed-set, Backbone=Base2023.03 | 55.4 | — | — | — | — | — | — | — | — | — | — | — | |
| OpenSeeD (T)Type=Open-vocabulary, Backbone=Tiny, Evaluation Protocol=Without fine-tuning2023.03 | 55.4 | — | — | — | — | — | — | — | — | — | — | — | |
| ODISElabel supervision=true, mask supervision=true, caption supervision=false2023.03 | 55.4 | — | — | — | — | — | — | — | — | — | 46 | 65.2 | |
| OMG-SegBackbone=ConvNeXt-XXL, Seg. Data=C, VIP, CM, V2024.03 | 55.4 | — | — | — | — | — | — | — | — | — | — | — | |
| CMT-DeepLabbackbone=Axial-R104-RFN, params=270.3M, pre-training=ImageNet-22K, training iterations=200k2022.06 | 55.3 | — | — | 61 | — | — | 46.6 | — | — | — | — | — | |
| CMT-DeepLabbackbone=Axial-R104-RFN, params=270.3M, pre-training=ImageNet-22K2022.06 | 55.1 | — | — | 60.6 | — | — | 46.8 | — | — | — | — | — | |
| Mask2FormerBackbone=Swin-B, Seg. Data=C2024.03 | 55.1 | — | — | — | — | — | — | — | — | — | — | — | |
| K-NetBackbone=Swin-L†, Epochs=36, #P (M)=208.92021.09 | 54.6 | — | — | 60.2 | — | — | 46 | — | — | — | — | — | |
| K-Netbackbone=Swin-L, pre-training=ImageNet-22K2022.06 | 54.6 | — | — | 60.2 | — | — | 46 | — | — | — | — | — | |
| K-NetBackbone=Swin-L2023.03 | 54.6 | — | — | 46 | — | — | 60.2 | — | — | — | — | — | |
| Mask2Former + DejaVuBackbone=Swin-T2023.03 | 54.3 | — | — | 44.9 | — | — | 60.5 | — | — | — | — | — | |
| CMT-DeepLabbackbone=Axial-R104, params=135.2M, pre-training=ImageNet-22K2022.06 | 54.1 | — | — | 58.8 | — | — | 47.1 | — | — | — | — | — | |
| MaskConverBackbone=ResNet50 [24], Params=57M, FLOPs=171B, FPS=19.6, Input Size=800 x 12002023.12 | 53.6 | — | — | 58.9 | — | — | 45.6 | — | — | — | — | — | |
| Mask2Former (T)Type=Closed-set, Backbone=Tiny2023.03 | 53.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2FormerBackbone=Swin-T2023.03 | 53.2 | — | — | 44 | — | — | 59.3 | — | — | — | — | — | |
| Mask2FormerBackbone=Swin-T, Param=47.4M2026.05 | 53.2 | 82.2 | 64.3 | — | — | — | — | — | — | — | — | — | |
| CMT-DeepLabbackbone=Axial-R50, params=94.9M, pre-training=ImageNet-22K2022.06 | 53 | — | — | 57.7 | — | — | 45.9 | — | — | — | — | — | |
| kMaX-DeepLabBackbone=ResNet50 [24], Params=57M, FLOPs=168B, FPS=22.8, Input Size=800 x 12002023.12 | 53 | — | — | 58.3 | — | — | 44.9 | — | — | — | — | — | |
| MaskFormerBackbone=Swin-L†, Epochs=300, #P (M)=212, #F (G)=7922021.09 | 52.7 | — | — | 58.5 | — | — | 44 | — | — | — | — | — | |
| MaskFormerbackbone=Swin-L, params=212M, pre-training=ImageNet-22K2022.06 | 52.7 | — | — | 58.5 | — | — | 44 | — | — | — | — | — | |
| MaskFormerBackbone=Swin-L2023.03 | 52.7 | — | — | 44 | — | — | 58.5 | — | — | — | — | — | |
| X-Decoder (T)Type=Open-vocabulary, Backbone=Tiny, Evaluation Protocol=Fine-tuned2023.03 | 52.6 | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2FormerBackbone=R50, Scale=800,1333, GPU=V1002023.03 | 51.9 | — | — | 57.7 | — | — | 43 | — | — | 8.6 | — | — | |
| Mask2FormerBackbone=ResNet50 [24], Params=44M, FLOPs=226B, FPS=8.6, Input Size=800 x 12002023.12 | 51.9 | — | — | 57.7 | — | — | 43 | — | — | — | — | — | |
| Panoptic FCNBackbone=Swin-L†, Epochs=362021.09 | 51.8 | — | — | 58.6 | — | — | 41.6 | — | — | — | — | — | |
| MaskFormerbackbone=Swin-B, params=102M, pre-training=ImageNet-22K2022.06 | 51.8 | — | — | 56.9 | — | — | 44.1 | — | — | — | — | — | |
| MaX-DeepLab-LBackbone=MaX-L, TTA=false, Params=451M, M-Adds=3692B2020.12 | 51.1 | — | — | 57 | — | — | 42.2 | — | — | — | — | — | |
| Max-Deeplab-LBackbone=Max-L [2], Epochs=54, #P (M)=451, #F (G)=18462021.09 | 51.1 | — | — | 57 | — | — | 42.2 | — | — | — | — | — | |
| MaX-DeepLab-Lbackbone=MaX-L, params=451M2022.06 | 51.1 | — | — | 57 | — | — | 42.2 | — | — | — | — | — | |
| MaX-DeeplabBackbone=Max-L2023.03 | 51.1 | — | — | 42.2 | — | — | 57 | — | — | — | — | — | |
| Panoptic SegFormerBackbone=R101, Epochs=24, #P (M)=69.9, #F (G)=2862021.09 | 50.6 | — | — | 55.5 | — | — | 43.2 | — | — | — | — | — | |
| Panoptic SegFormerBackbone=R50, Epochs=24, #P (M)=51, #F (G)=2142021.09 | 49.6 | — | — | 54.4 | — | — | 42.4 | — | — | — | — | — | |
| K-Netbackbone=R101-FPN2022.06 | 49.6 | — | — | 55.1 | — | — | 41.4 | — | — | — | — | — | |
| PanSegFormerBackbone=R50, Scale=800,13332023.03 | 49.6 | — | — | 54.4 | — | — | 42.4 | — | — | — | — | — | |
| Panoptic SegFormerBackbone=ResNet50 [24], Params=51M, FLOPs=214B, FPS=7.8, Input Size=800 x 12002023.12 | 49.6 | — | — | 54.4 | — | — | 42.4 | — | — | — | — | — | |
| CMT-DeepLabBackbone=ResNet50 [24], Input Size=800 x 12002023.12 | 48.5 | — | — | — | — | — | — | — | — | — | — | — | |
| MaX-DeepLab-SBackbone=MaX-S, TTA=false, Params=61.9M, M-Adds=324B2020.12 | 48.4 | — | — | 53 | — | — | 41.5 | — | — | — | — | — | |
| Max-Deeplab-SBackbone=Max-S [2], Epochs=54, #P (M)=61.9, #F (G)=1622021.09 | 48.4 | — | — | 53 | — | — | 41.5 | — | — | — | — | — | |
| MaX-DeepLab-Sbackbone=MaX-S, params=61.9M2022.06 | 48.4 | — | — | 53 | — | — | 41.5 | — | — | — | — | — | |
| MaX-DeeplabBackbone=Max-S2023.03 | 48.4 | — | — | 41.5 | — | — | 53 | — | — | — | — | — | |
| Max-DeepLabBackbone=Max-S, Scale=800,1333, GPU=V1002023.03 | 48.4 | — | — | 53 | — | — | 41.5 | — | — | 7.6 | — | — | |
| YOSOBackbone=R50, Scale=800,1333, GPU=V1002023.03 | 48.4 | — | — | 53.5 | — | — | 40.8 | — | — | 23.6 | — | — | |
| K-Netbackbone=R101-FPN-DCN2022.06 | 48.3 | — | — | 54 | — | — | 39.7 | — | — | — | — | — | |
| Panoptic SegFormerBackbone=R50, Epochs=12, #P (M)=51, #F (G)=2142021.09 | 48 | — | — | 52.3 | — | — | 41.5 | — | — | — | — | — | |
| MaskFormerBackbone=Swin-T2023.03 | 47.7 | — | — | 41.7 | — | — | 51.7 | — | — | — | — | — | |
| MaskFormerBackbone=R101, Epochs=300, #P (M)=64, #F (G)=2482021.09 | 47.6 | — | — | 52.5 | — | — | 40.3 | — | — | — | — | — | |
| K-NetBackbone=R50, Epochs=362021.09 | 47.1 | — | — | 51.7 | — | — | 40.3 | — | — | — | — | — | |
| K-NetBackbone=R50-FPN, Scale=800,13332023.03 | 47.1 | — | — | 51.7 | — | — | 40.3 | — | — | — | — | — | |
| K-NetBackbone=ResNet50 [24], Input Size=800 x 12002023.12 | 47.1 | — | — | 51.7 | — | — | 40.3 | — | — | — | — | — | |
| MaskFormerBackbone=R50, Epochs=300, #P (M)=45, #F (G)=1812021.09 | 46.5 | — | — | 51 | — | — | 39.8 | — | — | — | — | — | |
| MaskFormerBackbone=R50, Scale=800,1333, GPU=V1002023.03 | 46.5 | — | — | 51 | — | — | 39.8 | — | — | 17.6 | — | — | |
| MaskFormerBackbone=ResNet50 [24], Params=45M, FLOPs=181B, FPS=17.6, Input Size=800 x 12002023.12 | 46.5 | — | — | 51 | — | — | 39.8 | — | — | — | — | — | |
| YOSOBackbone=R50, Scale=512,800, GPU=V1002023.03 | 46.4 | — | — | 50.7 | — | — | 40 | — | — | 45.6 | — | — | |
| Panoptic-DeepLabBackbone=SWideRNet-(1, 1, 4), MS=true2020.11 | 45.8 | — | — | 51 | — | — | 38 | — | — | — | — | — | |
| ODISElabel supervision=false, mask supervision=true, caption supervision=true2023.03 | 45.6 | — | — | — | — | — | — | — | — | — | 38.4 | 52.4 | |
| MambaPanopticBackbone=SegMan, Param=35.7M2026.05 | 45.6 | 81.5 | 54.7 | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=SWideRNet-(1, 1, 4), MS=false2020.11 | 45.3 | — | — | 51.5 | — | — | 36.1 | — | — | — | — | — | |
| DETRBackbone=RN-101, TTA=false, Params=61.8M, M-Adds=314B2020.12 | 45.1 | — | — | 50.5 | — | — | 37 | — | — | — | — | — | |
| DETRBackbone=R101, Epochs=325, #P (M)=61.8, #F (G)=3062021.09 | 45.1 | — | — | 50.5 | — | — | 37 | — | — | — | — | — | |
| Panoptic FCNBackbone=Res50-FPN, Device=V100, FPS=9.2, Architecture type=box-free, Enhanced version=true2020.12 | 44.3 | 80.7 | 53 | 50 | 83.4 | 59.3 | 35.6 | 76.7 | 43.5 | — | — | — | |
| PanopticFCNBackbone=R50-FPN, Scale=800,1333, GPU=V1002023.03 | 44.3 | — | — | 50 | — | — | 35.6 | — | — | 9.2 | — | — | |
| Panoptic FCNBackbone=ResNet50 [24], FPS=9.2, Input Size=800 x 12002023.12 | 44.3 | — | — | 50 | — | — | 35.6 | — | — | — | — | — | |
| Axial-DeepLab-LBackbone=Axial-ResNet-L, Multi-scale=true, Params=44.9M, M-Adds=3867.7B2020.03 | 43.9 | — | — | 48.6 | — | — | 36.8 | — | — | — | — | — | |
| Axial-DeepLabBackbone=Axial-ResNet-L, MS=true2020.11 | 43.9 | — | — | 48.6 | — | — | 36.8 | — | — | — | — | — | |
| Axial-DeepLab-LBackbone=AX-L, TTA=true, Params=44.9M, M-Adds=3868B2020.12 | 43.9 | — | — | 48.6 | — | — | 36.8 | — | — | — | — | — | |
| SOGNetbackbone=ResNet-50, Multi-scale testing=false, flipping=false2019.11 | 43.7 | — | — | 50.6 | — | — | 33.2 | — | — | — | — | — | |
| Panoptic FCNBackbone=Res50-FPN, Device=V100, FPS=12.5, Architecture type=box-free2020.12 | 43.6 | 80.6 | 52.6 | 49.3 | 82.6 | 58.9 | 35 | 77.6 | 42.9 | — | — | — | |
| Panoptic FCNBackbone=R50, Epochs=36, #P (M)=37, #F (G)=2442021.09 | 43.6 | — | — | 49.3 | — | — | 35 | — | — | — | — | — | |
| Axial-DeepLab-LBackbone=Axial-ResNet-L, Multi-scale=false, Params=44.9M, M-Adds=343.9B2020.03 | 43.4 | — | — | 48.5 | — | — | 35.6 | — | — | — | — | — | |
| UnifyingBackbone=Res50-FPN, Architecture type=box-based2020.12 | 43.4 | 79.6 | 53 | 48.6 | — | — | 35.5 | — | — | — | — | — | |
| Axial-DeepLab-LBackbone=AX-L, TTA=false, Params=44.9M, M-Adds=344B2020.12 | 43.4 | — | — | 48.5 | — | — | 35.6 | — | — | — | — | — | |
| Unify. PSBackbone=R-50, Mask selection=true2021.06 | 43.4 | — | — | 48.6 | — | — | 35.5 | — | — | — | — | — | |
| DETRBackbone=R50, Epochs=325, #P (M)=42.9, #F (G)=2482021.09 | 43.4 | — | — | 48.2 | — | — | 36.3 | — | — | — | — | — | |
| DETRBackbone=ResNet50 [24], Input Size=800 x 12002023.12 | 43.4 | — | — | — | — | — | — | — | — | — | — | — | |
| UnifyingBackbone=ResNet50 [24], Input Size=800 x 12002023.12 | 43.3 | — | — | 48.6 | — | — | 35.5 | — | — | — | — | — | |
| UPSNetBackbone=ResNet-50, Flip=true, MS=true2019.11 | 43.2 | — | — | 49.1 | — | — | 34.1 | — | — | — | — | — | |
| Graphonomy (Panoptic)2021.01 | 43.2 | — | — | 49.8 | — | — | 33.4 | — | — | — | — | — | |
| UPSNetBackbone=RN-50, TTA=true2020.12 | 43.2 | — | — | 49.1 | — | — | 34.1 | — | — | — | — | — | |
| BGRNetBackbone=R50-FPN, Scale=800,13332023.03 | 43.2 | — | — | 49.8 | — | — | 33.4 | — | — | — | — | — | |
| Detectron2Backbone=ResNet-101, Flip=false, MS=false2019.11 | 43 | — | — | — | — | — | — | — | — | — | — | — | |
| Detectron2Backbone=RN-101, TTA=false2020.12 | 43 | — | — | — | — | — | — | — | — | — | — | — | |
| Axial-DeepLab-MBackbone=Axial-ResNet-M, Multi-scale=false, Params=25.9M, M-Adds=209.9B2020.03 | 42.9 | — | — | 47.6 | — | — | 35.8 | — | — | — | — | — | |
| Panoptic FCNBackbone=Res50-FPN, Input size=600, Device=V100, FPS=16.8, Architecture type=box-free2020.12 | 42.8 | 80.6 | 51.6 | 47.9 | 82.6 | 57.2 | 35.1 | 77.4 | 43.1 | — | — | — | |
| Panoptic FPNBackbone=LVT, Params (M)=5.4, FLOPs (G)=56.4, FPS (1333, 800)=20.42021.12 | 42.8 | — | — | 49.5 | — | — | 32.6 | — | — | — | — | — | |
| UPSNetbackbone=ResNet-50, Multi-scale testing=false, flipping=false2019.11 | 42.5 | — | — | 48.5 | — | — | 33.4 | — | — | — | — | — | |
| UPSNetBackbone=ResNet-50, Flip=false, MS=false2019.11 | 42.5 | — | — | 48.5 | — | — | 33.4 | — | — | — | — | — |