Semantic Segmentation on ADE20K v1 (val)
55.7mIoUFD-CLIP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FD-CLIPBackbone=ViT-L, Resolution=224x224, Feature Distillation=true2022.05 | 55.7 | — | — | |
| CLIPBackbone=ViT-L, Resolution=224x224, Feature Distillation=false2022.05 | 53.5 | — | — | |
| 4M-LPre-training data=CC12M, Data aug.=false, Extra labels=true2023.12 | 53.4 | — | — | |
| FD-CLIPBackbone=ViT-B, Resolution=224x224, Feature Distillation=true2022.05 | 52.8 | — | — | |
| TinyMIMBackbone=ViT-B/16, Pretraining Epochs=300, Teacher=MAE-ViT-L, Intermediate fine-tuning on ImageNet-1K=true2023.01 | 52.6 | — | — | |
| TinyMIMBackbone=ViT-B/16, Pretraining Epochs=300, Teacher=MAE-ViT-L, Intermediate fine-tuning on ImageNet-1K=false2023.01 | 52.2 | — | — | |
| DeiT III LPre-training data=IN-21K, Data aug.=true, Extra labels=false2023.12 | 52 | — | — | |
| MAE LPre-training data=IN-1K, Data aug.=true, Extra labels=false2023.12 | 51.8 | — | — | |
| RADIOFamily=RADIO, Model=ViT-H/14, Resolution=512, Evaluation Protocol=linear probing2023.12 | 51.3 | — | — | |
| HRViT-b3Backbone=HRViT-b3, Decoder Head=SegFormer Head, #Param. (M)=28.7, GFLOPs=67.92021.11 | 50.2 | — | — | |
| CAEBackbone=ViT-B/16, Pretraining Epochs=1600, Teacher=DALL-E2023.01 | 50.2 | — | — | |
| 4M-BPre-training data=CC12M, Data aug.=false, Extra labels=true2023.12 | 50.1 | — | — | |
| HRViT-b3Backbone=HRViT-b3, Decoder Head=UperNet Head, #Param. (M)=55.4, GFLOPs=2362021.11 | 50.04 | — | — | |
| CSWin-SBackbone=CSWin-S, Decoder Head=UperNet Head, #Param. (M)=64.6, GFLOPs=2472021.11 | 50 | — | — | |
| CSWin-SBackbone=CSWin-S, Decoder Head=SegFormer Head, #Param. (M)=37.3, GFLOPs=78.12021.11 | 49.93 | — | — | |
| CLIPBackbone=ViT-B, Resolution=224x224, Feature Distillation=false2022.05 | 49.5 | — | — | |
| MiT-B3Backbone=MiT-B3, Decoder Head=SegFormer Head, #Param. (M)=47.3, GFLOPs=79.02021.11 | 49.4 | — | — | |
| CSWin-TBackbone=CSWin-T, Decoder Head=UperNet Head, #Param. (M)=59.9, GFLOPs=2342021.11 | 49.3 | — | — | |
| HRViT-b2Backbone=HRViT-b2, Decoder Head=UperNet Head, #Param. (M)=49.7, GFLOPs=2332021.11 | 49.1 | — | — | |
| DeiT III BPre-training data=IN-21K, Data aug.=true, Extra labels=false2023.12 | 49 | — | — | |
| FD-EsViTBackbone=Swin-B, Resolution=224x224, Feature Distillation=true2022.05 | 48.9 | — | — | |
| TinyMIMBackbone=ViT-S/16, Pretraining Epochs=300, Teacher=TinyMIM-ViT-B, Intermediate fine-tuning on ImageNet-1K=true2023.01 | 48.9 | — | — | |
| HRViT-b2Backbone=HRViT-b2, Decoder Head=SegFormer Head, #Param. (M)=20.8, GFLOPs=28.02021.11 | 48.76 | — | — | |
| SdAEBackbone=ViT-B/16, Pretraining Epochs=300, Teacher=EMA2023.01 | 48.6 | — | — | |
| PeCoBackbone=ViT-B/16, Pretraining Epochs=800, Teacher=VQGAN2023.01 | 48.5 | — | — | |
| TinyMIMBackbone=ViT-S/16, Pretraining Epochs=300, Teacher=TinyMIM-ViT-B, Intermediate fine-tuning on ImageNet-1K=false2023.01 | 48.4 | — | — | |
| SimMIMBackbone=Swin-B, Resolution=224x224, Feature Distillation=false2022.05 | 48.3 | — | — | |
| MAEBackbone=ViT-B, Resolution=224x224, Feature Distillation=false2022.05 | 48.1 | — | — | |
| MAEBackbone=ViT-B/16, Pretraining Epochs=1600, Teacher=Pixel2023.01 | 48.1 | — | — | |
| FD-DeiTBackbone=ViT-B, Resolution=224x224, Feature Distillation=true2022.05 | 48 | — | — | |
| CSWin-TBackbone=CSWin-T, Decoder Head=SegFormer Head, #Param. (M)=22.4, GFLOPs=28.32021.11 | 47.88 | — | — | |
| Twins-BBackbone=Twins-B, Decoder Head=UperNet Head, #Param. (M)=88.5, GFLOPs=2482021.11 | 47.7 | — | — | |
| FD-DINOBackbone=ViT-B, Resolution=224x224, Feature Distillation=true2022.05 | 47.7 | — | — | |
| Swin-SBackbone=Swin-S, Decoder Head=UperNet Head, #Param. (M)=81.3, GFLOPs=2532021.11 | 47.6 | — | — | |
| SimMIMBackbone=ViT-B, Resolution=224x224, Feature Distillation=false2022.05 | 47.6 | — | — | |
| EsViTBackbone=Swin-B, Resolution=224x224, Feature Distillation=false2022.05 | 47.3 | — | — | |
| ScratchBackbone=ViT-B/16, Pretraining Epochs=300, Teacher=Label2023.01 | 47.2 | — | — | |
| HRViT-b1Backbone=HRViT-b1, Decoder Head=UperNet Head, #Param. (M)=35.9, GFLOPs=2192021.11 | 47.19 | — | — | |
| BEITBackbone=ViT-B, Resolution=224x224, Feature Distillation=false2022.05 | 47.1 | — | — | |
| DeiTBackbone=ViT-B, Resolution=224x224, Feature Distillation=false2022.05 | 47 | — | — | |
| PeCoBackbone=ViT-B/16, Pretraining Epochs=300, Teacher=VQGAN2023.01 | 46.7 | — | — | |
| MiT-B2Backbone=MiT-B2, Decoder Head=SegFormer Head, #Param. (M)=27.5, GFLOPs=62.42021.11 | 46.5 | — | — | |
| Twins-SBackbone=Twins-S, Decoder Head=UperNet Head, #Param. (M)=54.4, GFLOPs=2192021.11 | 46.2 | — | — | |
| DINOBackbone=ViT-B, Resolution=224x224, Feature Distillation=false2022.05 | 46.2 | — | — | |
| MultiMAE BPre-training data=IN-1K, Data aug.=true, Extra labels=true2023.12 | 46.2 | — | — | |
| MAE BPre-training data=IN-1K, Data aug.=true, Extra labels=false2023.12 | 46.1 | — | — | |
| HRViT-b1Backbone=HRViT-b1, Decoder Head=SegFormer Head, #Param. (M)=8.2, GFLOPs=14.62021.11 | 45.88 | — | — | |
| BeiTBackbone=ViT-B/16, Pretraining Epochs=800, Teacher=DALL-E2023.01 | 45.6 | — | — | |
| DINOBackbone=ViT-S/16, Pretraining Epochs=1600, Teacher=EMA2023.01 | 45.3 | — | — | |
| TinyMIM*-TParam (M)=5.8, Flops (G)=1.32023.01 | 45 | — | — | |
| TinyMIM*Backbone=ViT-T/16, Pretraining Epochs=300, Teacher=TinyMIM-ViT-S, Intermediate fine-tuning on ImageNet-1K=true, Fine-tuning duration=1000 epochs with DeiT-style distillation2023.01 | 45 | — | — | |
| TinyMIMBackbone=ViT-T/16, Pretraining Epochs=300, Teacher=TinyMIM-ViT-S, Intermediate fine-tuning on ImageNet-1K=true2023.01 | 44.6 | — | — | |
| Swin-TBackbone=Swin-T, Decoder Head=UperNet Head, #Param. (M)=59.9, GFLOPs=2342021.11 | 44.5 | — | — | |
| CSWin-TiBackbone=CSWin-Ti, Decoder Head=UperNet Head, #Param. (M)=33.7, GFLOPs=2162021.11 | 44.41 | — | — | |
| TinyMIMBackbone=ViT-T/16, Pretraining Epochs=300, Teacher=TinyMIM-ViT-S, Intermediate fine-tuning on ImageNet-1K=false2023.01 | 44 | — | — | |
| MoCoBackbone=ViT-S/16, Pretraining Epochs=1600, Teacher=EMA2023.01 | 43.9 | — | — | |
| MobileViTv3-SParam (M)=4.8, Flops (G)=1.82023.01 | 43.1 | — | — | |
| ScratchBackbone=ViT-S/16, Pretraining Epochs=300, Teacher=Label2023.01 | 43.1 | — | — | |
| 4M-B (RGB → CLIP only)Pre-training data=CC12M, Data aug.=false, Extra labels=true2023.12 | 43 | — | — | |
| MAEBackbone=ViT-S/16, Pretraining Epochs=1600, Teacher=Pixel2023.01 | 42.8 | — | — | |
| MobileViTv1-SParam (M)=4.9, Flops (G)=2.02023.01 | 42.7 | — | — | |
| MiT-B1Backbone=MiT-B1, Decoder Head=SegFormer Head, #Param. (M)=13.7, GFLOPs=15.92021.11 | 42.2 | — | — | |
| EdgeViT-XSParam (M)=6.4, Flops (G)=1.12023.01 | 42.1 | — | — | |
| CSWin-TiBackbone=CSWin-Ti, Decoder Head=SegFormer Head, #Param. (M)=5.9, GFLOPs=11.42021.11 | 41.43 | — | — | |
| SwinParam (M)=8.8, Flops (G)=1.22023.01 | 40.4 | — | — | |
| PVT-TParam (M)=13.0, Flops (G)=1.92023.01 | 39.8 | — | — | |
| MoCoBackbone=ViT-T/16, Pretraining Epochs=1600, Teacher=EMA2023.01 | 39.3 | — | — | |
| CiT-TParam (M)=5.5, Flops (G)=1.32023.01 | 38.5 | — | — | |
| SAM-CLIPFamily=SAM-CLIP, Model=ViTDet-B/16, Resolution=10242023.12 | 38.4 | — | — | |
| 4M-B (RGB → RGB only)Pre-training data=CC12M, Data aug.=false, Extra labels=false2023.12 | 38.3 | — | — | |
| DeiT-TParam (M)=5.5, Flops (G)=1.32023.01 | 38 | — | — | |
| ScratchBackbone=ViT-T/16, Pretraining Epochs=300, Teacher=Label2023.01 | 38 | — | — | |
| MAEBackbone=ViT-T/16, Pretraining Epochs=1600, Teacher=Pixel2023.01 | 37.6 | — | — | |
| MiT-B0Backbone=MiT-B0, Decoder Head=SegFormer Head, #Param. (M)=3.8, GFLOPs=8.42021.11 | 37.4 | — | — | |
| DFN CLIPFamily=DFN CLIP, Model=ViT-H/142023.12 | 31.7 | — | — | |
| SAMFamily=SAM, Model=ViTDet-H/162023.12 | 28.2 | — | — | |
| DeepLab V3+Backbone=R50, #P=44, #F=1772022.04 | — | 44 | 44.9 | |
| FASegBackbone=R50, #P=51, #F=722022.04 | — | 48.3 | 49.3 | |
| FASegBackbone=Swin-T, #P=54, #F=752022.04 | — | 49.6 | 51.3 | |
| FASegBackbone=Swin-B, #P=113, #F=2252022.04 | — | 55 | 56 | |
| FASegBackbone=Swin-L, #P=222, #F=4052022.04 | — | 56.3 | 57.7 | |
| Mask2formerBackbone=R50, #P=44, #F=712022.04 | — | 47.2 | 49.2 | |
| Mask2formerBackbone=Swin-T, #P=47, #F=742022.04 | — | 47.7 | 49.6 | |
| Mask2formerBackbone=Swin-B, #P=107, #F=2232022.04 | — | 53.9 | 55.1 | |
| Mask2formerBackbone=Swin-L, #P=215, #F=4032022.04 | — | 56.1 | 57.3 | |
| MaskformerBackbone=R50, #P=41, #F=532022.04 | — | 44.5 | 46.7 | |
| MaskformerBackbone=Swin-T, #P=42, #F=552022.04 | — | 46.7 | 48.8 | |
| MaskformerBackbone=Swin-B, #P=102, #F=1952022.04 | — | 52.7 | 53.9 | |
| MaskformerBackbone=Swin-L, #P=212, #F=3752022.04 | — | 54.1 | 55.6 | |
| PFDBackbone=R50, #P=74, #F=612022.04 | — | 45.6 | 48.3 | |
| PFDBackbone=Swin-T, #P=74, #F=652022.04 | — | 48.3 | 49.6 | |
| PFDBackbone=Swin-B, #P=123, #F=2062022.04 | — | 54.1 | 55.3 | |
| PFDBackbone=Swin-L, #P=242, #F=3852022.04 | — | 56 | 57.2 | |
| SenFormerBackbone=R50, #P=144, #F=1792022.04 | — | 44.7 | 45.2 | |
| SenFormerBackbone=Swin-T, #P=144, #F=1792022.04 | — | 46 | — | |
| SenFormerBackbone=Swin-B, #P=204, #F=2422022.04 | — | 51.8 | — | |
| SenFormerBackbone=Swin-L, #P=314, #F=5462022.04 | — | 53.1 | 54.2 | |
| UperNetBackbone=R50, #P=67, #F=2382022.04 | — | 42.1 | — | |
| UperNetBackbone=Swin-T, #P=60, #F=2362022.04 | — | 44.4 | 46.1 | |
| UperNetBackbone=Swin-L, #P=234, #F=6472022.04 | — | 52.1 | 53.5 |