Open Vocabulary Semantic Segmentation on Pascal Context PC-59
68.68mIoUOracle
Evaluation results
| Method | Links | |
|---|---|---|
| Oracleevaluation protocol=oracle2025.03 | 68.68 | |
| ESC-NetVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 65.6 | |
| SemLAfusion strategy=LoRA merging, tau=0.05, K=5, evaluation protocol=leave-one-out2025.03 | 64.75 | |
| MROVSegVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff2024.12 | 64.1 | |
| SemLAfusion strategy=Late Fusion2025.03 | 63.87 | |
| CAT-Segevaluation protocol=zero-shot2025.03 | 63.33 | |
| CAT-SegVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff2024.12 | 63.3 | |
| CAT-SegVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 63.3 | |
| Uniformfusion strategy=Late Fusion2025.03 | 63.24 | |
| Uniform mergingfusion strategy=LoRA merging2025.03 | 63.06 | |
| SMARTVLM=CLIP ViT-L/14, Training Dataset=COCO Panoptic2024.12 | 62.4 | |
| SEDVLM=CLIP ConvNeXt-L, Training Dataset=COCO-Stuff2024.12 | 60.9 | |
| SEDVLM=ConvNeXt-L, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 60.6 | |
| MAFTP w/ MaskAdapterVLM=CLIP ConvNeXt-L, Training Dataset=COCO-Stuff2024.12 | 60.4 | |
| EBSegVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff2024.12 | 60.2 | |
| EBSegVLM=CLIP ViT-L/14, Additional Backbone=SAM ViT-B, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 60.2 | |
| FC-CLIP w/ MaskAdapterVLM=CLIP ConvNeXt-L, Training Dataset=COCO Panoptic2024.12 | 59.7 | |
| GBAVLM=CLIP ConvNeXt-L, Training Dataset=COCO Panoptic2024.12 | 59.6 | |
| MAFTPVLM=CLIP ConvNeXt-L, Training Dataset=COCO-Stuff2024.12 | 59.5 | |
| MAFT+VLM=ConvNeXt-L, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 59.4 | |
| SCANVLM=CLIP ViT-L/14, Additional Backbone=Swin-B, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 59.3 | |
| ESC-NetVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 59 | |
| MAFTVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 59 | |
| MAFTP w/ MaskAdapterVLM=CLIP ConvNeXt-B, Training Dataset=COCO-Stuff2024.12 | 58.4 | |
| FC-CLIPVLM=CLIP ConvNeXt-L, Training Dataset=COCO Panoptic2024.12 | 58.4 | |
| SCANVLM=CLIP ViT-B/16, Additional Backbone=Swin-B, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 58.4 | |
| USEVLM=CLIP ViT-L/14, Additional Backbone=DINOv2, SAM, Training Dataset=COCO-Stuff, Additional Dataset=true2024.11 | 58 | |
| SANVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff2024.12 | 57.7 | |
| SANVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 57.7 | |
| CAT-SegVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff2024.12 | 57.5 | |
| MAFTPVLM=CLIP ConvNeXt-B, Training Dataset=COCO-Stuff2024.12 | 57.5 | |
| CAT-SegVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 57.5 | |
| SEDVLM=CLIP ConvNeXt-B, Training Dataset=COCO-Stuff2024.12 | 57.3 | |
| SEDVLM=ConvNeXt-B, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 57.3 | |
| ODISEVLM=CLIP ViT-L/14, Additional Backbone=Stable Diffusion, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 57.3 | |
| SEDVLM=CLIP ConvNeXt-B, Fine-tuning Space=E2026.05 | 57.3 | |
| HyRoVLM=CLIP ViT-B/16, Fine-tuning Space=H2026.05 | 57.3 | |
| HyperCLIPVLM=CLIP ViT-B/16, Fine-tuning Space=H2026.05 | 57.1 | |
| EBSegVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff2024.12 | 56.7 | |
| EBSegVLM=CLIP ViT-B/16, Additional Backbone=SAM ViT-B, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 56.7 | |
| OvSegVLM=CLIP ViT-L/14, Training Dataset=COCO-Stuff2024.12 | 55.7 | |
| OVSegVLM=CLIP ViT-L/14, Additional Backbone=Swin-B, Training Dataset=COCO-Stuff, Additional Dataset=true2024.11 | 55.7 | |
| ODISEVLM=Stable Diffusion, Training Dataset=COCO Panoptic2024.12 | 55.3 | |
| SANVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff2024.12 | 53.8 | |
| SANVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 53.8 | |
| SANVLM=CLIP ViT-B/16, Additional Backbone=Side Adapter, Fine-tuning Space=E2026.05 | 53.8 | |
| OvSegVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff2024.12 | 53.3 | |
| OVSegVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101c, Training Dataset=COCO-Stuff, Additional Dataset=true2024.11 | 53.3 | |
| OVSegVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101c, Fine-tuning Space=E2026.05 | 53.3 | |
| DeOPVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff2024.12 | 48.8 | |
| DeOPVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101c, Training Dataset=COCO-Stuff-156, Additional Dataset=false2024.11 | 48.8 | |
| OpenSegVLM=ALIGN, Training Dataset=COCO Panoptic+Loc. Narr.2024.12 | 48.2 | |
| ZSsegVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 47.7 | |
| ZSsegVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101, Fine-tuning Space=E2026.05 | 47.7 | |
| ZegFormerVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff2024.12 | 45.5 | |
| OpenSegVLM=ALIGN, Additional Backbone=EfficientNet-B7, Training Dataset=COCO Panoptic, Additional Dataset=true2024.11 | 44.8 | |
| ZegFormerVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101, Training Dataset=COCO-Stuff-156, Additional Dataset=false2024.11 | 42.8 | |
| ZegFormerVLM=CLIP ViT-B/16, Additional Backbone=ResNet-101, Fine-tuning Space=E2026.05 | 42.8 | |
| FC-CLIPVLM=ConvNeXt-L, Training Dataset=COCO Panoptic, Additional Dataset=false2024.11 | 42.4 | |
| SPARBackbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=AnyUp [45]2026.04 | 41.5 | |
| ZegCLIPVLM=CLIP ViT-B/16, Training Dataset=COCO-Stuff-156, Additional Dataset=false2024.11 | 41.2 | |
| ZegCLIPVLM=CLIP ViT-B/16, Fine-tuning Space=E2026.05 | 41.2 | |
| FreeDAPub. & Year=CVPR’24, Extra data=COCO Captions, Extra backbone=DINOv2 (ViT-B/14), Training free=✓, Mask Refinement=true2026.03 | 40.2 | |
| OpenSegVLM=ALIGN, Additional Backbone=ResNet-101, Training Dataset=COCO Panoptic, Additional Dataset=true2024.11 | 40.1 | |
| OpenSegVLM=ALIGN, Additional Backbone=ResNet-101, Fine-tuning Space=E2026.05 | 40.1 | |
| LPOSS + SPARBackbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 39.8 | |
| ProxyCLIP*Pub. & Year=ECCV’24, Extra data=✗, Extra backbone=DINOv2† (ViT-B/14), Training free=✓, Mask Refinement=true2026.03 | 38.9 | |
| PEARLExtra data=✗, Extra backbone=✗, Training free=✓, Mask Refinement=false2026.03 | 38.6 | |
| NACLIP*Pub. & Year=WACV’25, Extra data=✗, Extra backbone=✗, Training free=✓, Mask Refinement=true2026.03 | 38.4 | |
| SFP*Pub. & Year=ICCV’25, Extra data=✗, Extra backbone=✗, Training free=✓, Mask Refinement=true2026.03 | 38.3 | |
| CASS*Pub. & Year=CVPR’25, Extra data=✗, Extra backbone=DINOv3 (ViT-B/16), Training free=✓, Mask Refinement=true2026.03 | 38.2 | |
| SPARBackbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 37.2 | |
| CLIP-DINOiserPub. & Year=ECCV’24, Extra data=ImageNet-1K, Extra backbone=DINOv1 (ViT-B/16), Training free=✗, Mask Refinement=true2026.03 | 37.1 | |
| SCLIP*Pub. & Year=ECCV’24, Extra data=✗, Extra backbone=✗, Training free=✓, Mask Refinement=true2026.03 | 36.1 | |
| LSeg+VLM=ALIGN, Additional Backbone=ResNet-101, Training Dataset=COCO-Stuff, Additional Dataset=false2024.11 | 36 | |
| CASS*Pub. & Year=CVPR’25, Extra data=✗, Extra backbone=DINOv2† (ViT-B/14), Training free=✓, Mask Refinement=true2026.03 | 35.9 | |
| LPOSS + Pre-tr.Backbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 35.2 | |
| Pre-trainedBackbone=DINOv3 - ViT-L-16, Inference=single-pass, Upsampler=AnyUp [45]2026.04 | 35.1 | |
| SPARBackbone=DINOv3 - ViT-L-16, Inference=single-pass, Upsampler=bilinear2026.04 | 35 | |
| SPARBackbone=DINOv3 - ViT-L-16, Inference=single-pass, Upsampler=AnyUp [45]2026.04 | 35 | |
| Pre-trainedBackbone=DINOv3 - ViT-L-16, Inference=single-pass, Upsampler=bilinear2026.04 | 34.8 | |
| LaVGPub. & Year=ECCV’24, Extra data=✗, Extra backbone=DINOv1 (ViT-B/8), Training free=✓, Mask Refinement=true2026.03 | 34.7 | |
| Pre-trainedBackbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=AnyUp [45]2026.04 | 34.5 | |
| Pre-trainedBackbone=DINOv3 - ViT-L-16, Inference=sliding-window, Upsampler=bilinear2026.04 | 34.4 | |
| Pre-trainedBackbone=SigLIP2 - ViT-B-16, Inference=sliding-window, Upsampler=bilinear2026.04 | 34 | |
| TCLPub. & Year=CVPR’23, Extra data=CC3M+CC12M, Extra backbone=✗, Training free=✗, Mask Refinement=true2026.03 | 33.9 | |
| SPARBackbone=OpenCLIP - ViT-B-16, Inference=single-pass, Upsampler=AnyUp [45]2026.04 | 32 | |
| SPARBackbone=OpenCLIP - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 30.3 | |
| Pre-trainedBackbone=OpenCLIP - ViT-B-16, Inference=single-pass, Upsampler=AnyUp [45]2026.04 | 26.9 | |
| Pre-trainedBackbone=OpenCLIP - ViT-B-16, Inference=sliding-window, Upsampler=bilinear2026.04 | 26.6 | |
| Pre-trainedBackbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 26.1 | |
| MaskCLIPPub. & Year=ECCV’22, Extra data=✗, Extra backbone=✗, Training free=✓, Mask Refinement=true2026.03 | 25.3 | |
| NaFlexBackbone=SigLIP2 - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 25.2 | |
| ReCoPub. & Year=NeurIPS’22, Extra data=ImageNet-1K, Extra backbone=✗, Training free=✓, Mask Refinement=true2026.03 | 24.7 | |
| SPNetAdditional Backbone=ResNet-101, Training Dataset=PASCAL VOC, Additional Dataset=false2024.11 | 24.3 | |
| GroupViTPub. & Year=CVPR’22, Extra data=CC12M+RedCaps, Extra backbone=✗, Training free=✗, Mask Refinement=true2026.03 | 23.8 | |
| Pre-trainedBackbone=OpenCLIP - ViT-B-16, Inference=single-pass, Upsampler=bilinear2026.04 | 21.4 | |
| ZS3NetAdditional Backbone=ResNet-101, Training Dataset=PASCAL VOC, Additional Dataset=false2024.11 | 19.4 | |
| ZS3NetAdditional Backbone=ResNet-101, Fine-tuning Space=E2026.05 | 19.4 |