Semantic Segmentation on COCO (val)
90.5mIoUTokenizer reconstruction*
Evaluation results
| Method | Links | |
|---|---|---|
| Tokenizer reconstruction*Resolution=224x224, Note=Performance upper bound estimate2023.12 | 90.5 | |
| OpenSeeD (L)Type=Open-vocabulary, Backbone=Large, Evaluation Protocol=Without fine-tuning2023.03 | 68.6 | |
| MaskDINO (L)Type=Closed-set, Backbone=Large2023.03 | 67.5 | |
| X-Decoder (L)Type=Open-vocabulary, Backbone=Large, Evaluation Protocol=Fine-tuned2023.03 | 67.5 | |
| Mask2Former (L)Type=Closed-set, Backbone=Large2023.03 | 67.4 | |
| OneFormer (L)Type=Closed-set, Backbone=Large2023.03 | 67.4 | |
| Mask2FormerBackbone=Swin-L, Seg. Data=C2024.03 | 67.4 | |
| Mask2Former (B)Type=Closed-set, Backbone=Base2023.03 | 67.1 | |
| Mask2FormerBackbone=Swin-L, Supervision=M2024.03 | 66.7 | |
| PSALMBackbone=Swin-B, Seg. Data=C, RC, CI2024.03 | 66.6 | |
| SEEMBackbone=DaViT-B, Seg. Data=C, L, RC2024.03 | 66.3 | |
| X-DecoderBackbone=DaViT-B, Seg. Data=C, L, RC2024.03 | 66 | |
| Mask2FormerBackbone=Swin-B, Seg. Data=C2024.03 | 65.1 | |
| OpenSeeD (T)Type=Open-vocabulary, Backbone=Tiny, Evaluation Protocol=Without fine-tuning2023.03 | 64 | |
| Mask2Former (T)Type=Closed-set, Backbone=Tiny2023.03 | 63.2 | |
| Mask2FormerBackbone=Swin-T, Param=47.4M2026.05 | 63.2 | |
| X-Decoder (T)Type=Open-vocabulary, Backbone=Tiny, Evaluation Protocol=Fine-tuned2023.03 | 62.4 | |
| DeepLabv3+Backbone=ResNet-101, Supervision=M2024.03 | 61.8 | |
| DHRBackbone=Swin-L, Supervision=I2024.03 | 56.8 | |
| FMA-WSSSBackbone=Swin-L, Supervision=I+C+S2024.03 | 55.4 | |
| DHRBackbone=ResNet-101, Supervision=I2024.03 | 53.9 | |
| CoSABackbone=Swin-B, Supervision=I2024.03 | 53.7 | |
| ODISEPre-training Dataset=COCO, Pre-training Supervision=mask2026.03 | 52.4 | |
| CoSABackbone=ResNet-101, Supervision=I2024.03 | 50.9 | |
| UniMatchBackbone=Xception-65, Labeled ratio=1/32 (3697)2022.08 | 49.8 | |
| MARSBackbone=ResNet-101, Supervision=I2024.03 | 49.4 | |
| DiCLIP (Ours)Supervision=I + L, Backbone=ViT-B2026.05 | 48.7 | |
| FMA-WSSSBackbone=ResNet-101, Supervision=I+C+S2024.03 | 48.6 | |
| UniMatchBackbone=Xception-65, Labeled ratio=1/64 (1849)2022.08 | 48.2 | |
| Mask2Former Swin-LZero-shot=true, Resolution=224x2242023.12 | 48 | |
| POTSupervision=I + L, Backbone=ViT-B2026.05 | 47.9 | |
| DiCLIP (w/o CRF)Supervision=I + L, Backbone=ViT-B2026.05 | 47.6 | |
| MoReSupervision=I, Backbone=ViT-B2026.05 | 47.4 | |
| WeakCLIPSupervision=I + L, Backbone=MiT-B2026.05 | 47.4 | |
| PSDPMSupervision=I + L, Backbone=RN1012026.05 | 47.2 | |
| PCRESupervision=I, Backbone=ViT-B2026.05 | 47.2 | |
| WeCLIPSupervision=I + L, Backbone=ViT-B2026.05 | 47.1 | |
| SSCSupervision=I, Backbone=RN1012026.05 | 47 | |
| WeakTrBackbone=DeiT-S, Supervision=I2024.03 | 46.9 | |
| CPALSupervision=I + L, Backbone=RN1012026.05 | 46.8 | |
| FFRSupervision=I, Backbone=ViT-B2026.05 | 46.8 | |
| SeCoSupervision=I, Backbone=ViT-B2026.05 | 46.7 | |
| MuP-VSSSupervision=I, Backbone=ViT-B2026.05 | 46.6 | |
| 4M-XLZero-shot=true, Resolution=224x2242023.12 | 46.5 | |
| RSEPMBackbone=ResNet-101, Supervision=I2024.03 | 46.4 | |
| MambaPanopticBackbone=SegMan, Param=35.7M2026.05 | 46.4 | |
| DGRMSupervision=I, Backbone=RN1012026.05 | 46.3 | |
| PC2SegBackbone=Xception-65, Labeled ratio=1/32 (3697)2022.08 | 46.1 | |
| MMSCTBackbone=Wide-ResNet-38, Supervision=I+C2024.03 | 45.9 | |
| 4M-LZero-shot=true, Resolution=224x2242023.12 | 45.8 | |
| Mask2Former Swin-B [P]Zero-shot=true, Resolution=224x224, Role=Pseudo labeler2023.12 | 45.7 | |
| Mat-labelSupervision=I + SA, Backbone=RN1012026.05 | 45.6 | |
| DiGSupervision=I, Backbone=WRN382026.05 | 45.5 | |
| Lin et al.Framework=Multi-stage, Sup.=I + T, Net.=DL-V22024.03 | 45.4 | |
| CLIP-ESBackbone=ResNet-101, Supervision=I+C2024.03 | 45.4 | |
| CLIP-ESSupervision=I + L, Backbone=RN1012026.05 | 45.4 | |
| ACRFramework=Multi-stage, Sup.=I, Net.=DL-V22024.03 | 45.3 | |
| ACRBackbone=Wide-ResNet-38, Supervision=I2024.03 | 45.3 | |
| MCTformer+Supervision=I, Backbone=RN382026.05 | 45.2 | |
| BECOBackbone=MiT-B2, Supervision=I2024.03 | 45.1 | |
| BECOSupervision=I, Backbone=RN1012026.05 | 45.1 | |
| ACRw/ saliency=false2023.08 | 45 | |
| ReCAMFramework=Multi-stage, Sup.=I, Net.=DL-V22024.03 | 45 | |
| SASBackbone=ResNet-101, Supervision=I2024.03 | 44.8 | |
| SANCEBackbone=ResNet-101, Supervision=I2024.03 | 44.7 | |
| PLDASupervision=I, Backbone=RN1012026.05 | 44.7 | |
| Mask2Former Swin-SZero-shot=true, Resolution=224x2242023.12 | 44.6 | |
| DuPLFramework=One-stage, Sup.=I, Net.=ViT-B, Pre-trained=ImageNet-21k2024.03 | 44.6 | |
| DuPLSupervision=I, Backbone=ViT-B2026.05 | 44.6 | |
| UniMatchBackbone=Xception-65, Labeled ratio=1/128 (925)2022.08 | 44.4 | |
| DIALSupervision=I + L, Backbone=ViT-B2026.05 | 44.4 | |
| USAGESupervision=I, Backbone=RN382026.05 | 44.3 | |
| L2GVenue=CVPR2022, w/ saliency=true2023.08 | 44.2 | |
| L2GFramework=Multi-stage, Sup.=I + S, Net.=DL-V22024.03 | 44.2 | |
| L2GBackbone=ResNet-101, Supervision=I+A2024.03 | 44.2 | |
| L2GSupervision=I + SA, Backbone=RN1012026.05 | 44.2 | |
| FPRSupervision=I, Backbone=RN1012026.05 | 44 | |
| RIBVenue=NeurIPS2020, w/ saliency=false2023.08 | 43.8 | |
| RIBBackbone=ResNet-101, Supervision=I2024.03 | 43.8 | |
| PC2SegBackbone=Xception-65, Labeled ratio=1/64 (1849)2022.08 | 43.7 | |
| PseudoSegBackbone=Xception-65, Labeled ratio=1/32 (3697)2022.08 | 43.6 | |
| SIPEVenue=CVPR2022, w/ saliency=false2023.08 | 43.6 | |
| PanopticFCNBackbone=ResNet50, Param=36.6M2026.05 | 43.6 | |
| DuPLFramework=One-stage, Sup.=I, Net.=ViT-B, Pre-trained=ImageNet-1k2024.03 | 43.5 | |
| QA-CLIMSBackbone=ResNet-101, Supervision=I+L2024.03 | 43.2 | |
| ToMaSupervision=I, Backbone=ViT-B2026.05 | 43.2 | |
| ESOLBackbone=ResNet1012022.09 | 42.6 | |
| ESOLVenue=NeurIPS2022, w/ saliency=false2023.08 | 42.6 | |
| ESOLFramework=Multi-stage, Sup.=I, Net.=DL-V22024.03 | 42.6 | |
| OCR+MCTformerCls=DeiT-S, Seg=V1-Res382022.11 | 42.5 | |
| OCRFramework=Multi-stage, Sup.=I, Net.=WR-382024.03 | 42.5 | |
| OCRBackbone=Wide-ResNet-38, Supervision=I2024.03 | 42.5 | |
| OCRSupervision=I, Backbone=RN382026.05 | 42.5 | |
| ToCoBackbone=ViT-B, Supervision=I2024.03 | 42.3 | |
| ToCoSupervision=I, Backbone=ViT-B2026.05 | 42.3 | |
| SupBaselineBackbone=Xception-65, Labeled ratio=1/32 (3697)2022.08 | 42.2 | |
| MCTformerCls=Res38, Seg=V1-Res382022.11 | 42 | |
| MCTformerCls=DeiT-S, Seg=V1-Res382022.11 | 42 | |
| MCTformerVenue=CVPR2022, w/ saliency=false2023.08 | 42 | |
| MCTformerFramework=Multi-stage, Sup.=I, Net.=WR-382024.03 | 42 |