Instance Segmentation on Cityscapes (val)
50.6APOneFormer
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OneFormerBackbone=InternImage-H [46], Coarse=false, number of parameters=1.1B, queries=250, Crop Size=512x10242025.05 | 50.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormer + ViT-PBackbone=InternImage-H [46], Coarse=true, number of parameters=1.4B, queries=250, Crop Size=512x10242025.05 | 50.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask TransfinerBackbone=R50-FPN2021.11 | 49.8 | — | 64.1 | — | — | 18 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RefineMaskBackbone=R50-FPN2021.11 | 49.2 | — | 63.3 | — | — | 17.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormer + ViT-PBackbone=ConvNeXt-L [34], Coarse=true, number of parameters=306M, queries=250, Crop Size=512x10242025.05 | 49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-XL [34], Coarse=false, number of parameters=372M, queries=250, Crop Size=512x1024, Pre-training=Mapillary Vistas2025.05 | 48.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-L [34], Coarse=false, number of parameters=220M, queries=250, Crop Size=512x1024, Pre-training=COCO and Objects3652025.05 | 48.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OpenSeeD+Backbone=Swin-L [32], Coarse=false, number of parameters=286M, queries=300, Crop Size=512x1024, Pre-training=COCO and Objects3652025.05 | 48.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=R50-FPN2021.11 | 47.7 | — | 57.9 | — | — | 16.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PointRendBackbone=R50-FPN2021.11 | 47.2 | — | 61.8 | — | — | 16.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=SWideRNet, Crop Size=1025x2049, Extra Data=true, Multi-Scale (PQ & AP)=true2022.11 | 46.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-XL+ [39], number of parameters=372M, GFLOPS=775G, number of queries=250, Crop Size=512x1024, Iters=90k, Training Strategy=Joint Training2022.11 | 46.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-XL, Number of parameters=372M, Crop Size=512x1024, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 46.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-L [39], number of parameters=220M, GFLOPS=497G, number of queries=250, Crop Size=512x1024, Iters=90k, Training Strategy=Joint Training2022.11 | 46.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-L, Number of parameters=220M, Crop Size=512x1024, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 46.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=ConvNeXt-L [34], Coarse=false, number of parameters=220M, queries=250, Crop Size=512x10242025.05 | 46.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BMask R-CNNBackbone=R50-FPN2021.11 | 46.2 | — | 62.6 | — | — | 15.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFF-BaseBackbone=AFF-Base, Segmentation Head=Mask2Former*, # Params=75.3M, Point cloud input=true, Seed=0, alpha=8, Pre-training=ImageNet-22K2023.04 | 46.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=Swin-L+ [38], number of parameters=219M, GFLOPS=543G, number of queries=250, Crop Size=512x1024, Iters=90k, Training Strategy=Joint Training2022.11 | 45.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=DINAT-L+ [21], number of parameters=223M, GFLOPS=450G, number of queries=250, Crop Size=512x1024, Iters=90k, Training Strategy=Joint Training2022.11 | 45.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=Swin-L, Number of parameters=219M, Crop Size=512x1024, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 45.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OneFormerBackbone=DINAT-L, Number of parameters=223M, Crop Size=512x1024, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 45.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MP-Formerbackbone=Swin-L2023.03 | 44.9 | — | 72.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PolyTransformtraining data=fine+COCO2019.12 | 44.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Axial-DeepLab-XLBackbone=Axial ResNet-XL, Number of parameters=173M, Crop Size=1025x2049, Extra Data=true, Multi-Scale (PQ & AP)=true2022.11 | 44.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| kMaX-DeepLab+Backbone=ConvNeXt-L [39], number of parameters=232M, GFLOPS=1673G, number of queries=256, Crop Size=1025x2049, Iters=60k, Training Strategy=Individual Training2022.11 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| kMaX-DeepLabBackbone=ConvNeXt-L, Number of parameters=232M, Crop Size=1025x2049, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFF-SmallBackbone=AFF-Small, Segmentation Head=Mask2Former*, # Params=42.6M, Point cloud input=true, Seed=0, alpha=8, Pre-training=ImageNet-1K2023.04 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=Swin-L, Pre-training=ImageNet-21k, Amount of labeled data=100%, Learning setting=Semi-supervised2023.08 | 43.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former-InstanceBackbone=Swin-L+ [38], number of parameters=216M, GFLOPS=507G, number of queries=200, Crop Size=512x1024, Iters=90k, Training Strategy=Individual Training2022.11 | 43.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former-InstanceBackbone=Swin-L, Number of parameters=216M, Crop Size=512x1024, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 43.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Formerbackbone=Swin-L2023.03 | 43.7 | — | 71.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-LargeBackbone=Swin-Large, Segmentation Head=Mask2Former, # Params=197M, Point cloud input=false, Seed=0, alpha=8, Pre-training=ImageNet-22K2023.04 | 43.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former-PanopticBackbone=Swin-L+ [38], number of parameters=216M, GFLOPS=514G, number of queries=200, Crop Size=512x1024, Iters=90k, Training Strategy=Individual Training2022.11 | 43.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former-PanopticBackbone=Swin-L, Number of parameters=216M, Crop Size=512x1024, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 43.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Former-PanopticBackbone=Swin-L [32], Coarse=false, number of parameters=216M, queries=200, Crop Size=512x10242025.05 | 43.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientPSBackbone=EfficientNet, Crop Size=1025x2049, Extra Data=true, Multi-Scale (PQ & AP)=true2022.11 | 43.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=ViT-L, Pre-training=DINOv2, Amount of labeled data=100%, Learning setting=Semi-supervised2023.08 | 42.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=Xception-71, Extra Data=Mapillary Vistas, MS=true2019.11 | 42.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=SWideRNet, Crop Size=1025x2049, Extra Data=true, Multi-Scale (PQ & AP)=false2022.11 | 42.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFF-TinyBackbone=AFF-Tiny, Segmentation Head=Mask2Former*, # Params=27M, Point cloud input=true, Seed=0, alpha=8, Pre-training=ImageNet-1K2023.04 | 42.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-BaseBackbone=Swin-Base, Segmentation Head=Mask2Former, # Params=88M, Point cloud input=false, Seed=0, alpha=8, Pre-training=ImageNet-22K2023.04 | 42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Axial-DeepLab-XLBackbone=Axial ResNet-XL, Number of parameters=123M, Crop Size=1025x2049, Extra Data=true, Multi-Scale (PQ & AP)=false2022.11 | 41.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-SmallBackbone=Swin-Small, Segmentation Head=Mask2Former, # Params=50M, Point cloud input=false, Seed=0, alpha=8, Pre-training=ImageNet-1K2023.04 | 41.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PANettraining data=fine+COCO2019.12 | 41.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAST (student)Dataset Fraction=30%2025.05 | 40.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PolySnake†Training data=fine, FPS=4.22023.01 | 40.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLab+Backbone=SWideRNet+ [8], number of parameters=536M, GFLOPS=10365G, Crop Size=1025x2049, Iters=60k, Training Strategy=Individual Training2022.11 | 40.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=SWideRNet, Crop Size=1025x2049, Extra Data=false, Multi-Scale (PQ & AP)=false2022.11 | 40.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=SWideRNet [10], Coarse=false, number of parameters=536M, Crop Size=1025×20492025.05 | 40.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AFF-MiniBackbone=AFF-Mini, Segmentation Head=Mask2Former*, # Params=6.75M, Point cloud input=true, Seed=0, alpha=8, Pre-training=ImageNet-1K2023.04 | 40 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MambaPanopticBackbone=SegMan, Param=35.7M2026.05 | 39.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PolySnakeTraining data=fine, FPS=4.82023.01 | 39.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MP-Formerbackbone=R502023.03 | 39.7 | — | 65.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-TinyBackbone=Swin-Tiny, Segmentation Head=Mask2Former, # Params=28M, Point cloud input=false, Seed=0, alpha=8, Pre-training=ImageNet-1K2023.04 | 39.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CleanBackbone=Swin-T2024.10 | 39.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=ResNet-50, Pre-training=ImageNet-1k, Amount of labeled data=100%, Learning setting=Semi-supervised2023.08 | 39.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=ViT-B, Pre-training=DINOv2, Amount of labeled data=100%, Learning setting=Semi-supervised2023.08 | 39.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=Swin-L, Pre-training=ImageNet-21k, Amount of labeled data=30%, Learning setting=Semi-supervised2023.08 | 39.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours-D3S2Backbone=ViT-B/16, Mode=Hybrid2026.05 | 39.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=Xception-71, Extra Data=Mapillary Vistas, Flip=true2019.11 | 39.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QueryInstBackbone=ResNet-502021.05 | 39.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CondInstBackbone=DCN-101-BiFPN, semantic_branch=true2021.05 | 39.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TASCNetBackbone=Xception-71, Extra Data=COCO, MS=true2019.11 | 39.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=ViT-L, Pre-training=DINOv2, Amount of labeled data=30%, Learning setting=Semi-supervised2023.08 | 39.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2FormerBackbone=Swin-T, Param=47.4M2026.05 | 39.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNet-101-M-COCOBackbone=ResNet-101, Pre-training=COCO, Testing Scale=Multi-scale2019.01 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TASCNet-multiscalePre-training=ImageNet+COCO, Backbone=ResNet-50, multi-scale=true2019.09 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNet-multiscalePre-training=ImageNet+COCO, Backbone=ResNet-101, multi-scale=true2019.09 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNetBackbone=Xception-71, Extra Data=COCO, MS=true2019.11 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| E2ECTraining data=Fine, backbone=DLA-34, fps=4.9, multi-component detection=true2022.03 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| E2ECTraining data=fine, FPS=4.92023.01 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=Xception-71, Extra Data=Mapillary Vistas2019.11 | 38.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DistillationDataset Fraction=30%2025.05 | 38.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Panoptic-DeepLabBackbone=Xception-71, MS=true2019.11 | 38.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EfficientPSBackbone=Custom, Mask selection=true2021.06 | 38.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Guided DistillationBackbone=Swin-L, Pre-training=ImageNet-21k, Amount of labeled data=20%, Learning setting=Semi-supervised2023.08 | 38.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DynaMaskBackbone=ResNet-50-FPN2023.03 | 38 | — | — | — | — | — | — | 14.8 | 35.1 | 57.6 | — | — | — | — | — | — | — | — | — | — | — | |
| Swin-B (IN21K)Backbone=Swin-B, Mode=FT2026.05 | 38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| S4MDataset Fraction=30%2025.05 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNet-COCOBackbone=ResNet-50, Pre-training=COCO, Testing Scale=Single-scale2019.01 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNetPre-training=ImageNet+COCO, Backbone=ResNet-101, multi-scale=false2019.09 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNetBackbone=Xception-71, Extra Data=COCO2019.11 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNettraining data=fine+COCO2019.12 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNetTraining data=Fine+COCO, backbone=ResNet50, fps=4.42022.03 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CondInstSupervision=Mask-supervised, Data=fine + COCO2022.10 | 37.8 | — | 63.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNetTraining data=fine + COCO, FPS=4.42023.01 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UPSNetBackbone=ResNet-502021.05 | 37.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask R-CNN + SharpContourBackbone=ResNet-50, Neck=FPN, Training Annotations=fine, Training Epochs=64, Training Scale=multi-scale2022.03 | 37.7 | — | — | — | — | — | — | 14.4 | 34.2 | 58.3 | — | — | — | — | — | — | — | — | — | — | — | |
| TASCNetPre-training=ImageNet+COCO, Backbone=ResNet-50, multi-scale=false2019.09 | 37.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TASCNetBackbone=Xception-71, Extra Data=COCO2019.11 | 37.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RefineMaskBackbone=R50-FPN, Training Epochs=64, Training Annotations=fine, Training Resolution=multi-scale [800, 1024], Evaluation Protocol=median of three runs2021.04 | 37.6 | — | — | — | — | — | — | 14.6 | 34 | 58.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RefineMaskBackbone=ResNet-50, Neck=FPN, Training Annotations=fine, Training Epochs=64, Training Scale=multi-scale2022.03 | 37.6 | — | — | — | — | — | — | 14.6 | 34 | 58.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RefineMaskBackbone=ResNet-50-FPN2023.03 | 37.6 | — | — | — | — | — | — | 14.6 | 34 | 58.1 | — | — | — | — | — | — | — | — | — | — | — | |
| CondInstBackbone=ResNet-502021.05 | 37.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask R-CNN (Baseline)Backbone=R50-FPN2021.11 | 37.4 | — | 61.5 | — | — | 11.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Deep SnakeTraining data=Fine, backbone=DLA-34, fps=4.6, multi-component detection=true2022.03 | 37.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mask2Formerbackbone=R502023.03 | 37.4 | — | 61.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSnakeTraining data=fine, FPS=4.62023.01 | 37.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CleanBackbone=ResNet502024.10 | 37.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |