Semantic Segmentation on NYU v2 (test)
59.13mIoUPanopticNDT (EMSANet-R34-NBt1D)
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PanopticNDT (EMSANet-R34-NBt1D)Protocol=Fine-tuning, LR=0.004, Selection=Best in run2023.09 | 59.13 | — | — | — | — | — | — | — | — | — | — | — | |
| DVEFormerBackbone=SwinV2-T-128, Extra Training Data=true, Evaluation Protocol=Linear probing2026.01 | 57.07 | — | — | — | — | — | — | — | 26.3 | — | — | — | |
| CMX-B5Year=2022, Backbone=Segformer-B5, Params/M=181, Real-Time=false, Multi-Scale inference (MS)=true, Input Resolution=480x6402023.09 | 56.9 | — | — | — | — | — | — | — | — | — | — | — | |
| CMNeXtBackbone=2x MiT-B42026.01 | 56.9 | — | — | — | — | — | — | — | 14.84 | — | — | — | |
| OmnivoreYear=2022, Backbone=Swin-L, Input Resolution=480x6402023.09 | 56.8 | — | — | — | — | — | — | — | — | — | — | — | |
| PanopticNDT (EMSANet-R34-NBt1D)Protocol=Application network, LR=0.00052023.09 | 56.55 | — | — | — | — | — | — | — | — | — | — | — | |
| EMSANetBackbone=2x ResNet34-NBt1D, Extra Training Data=true, Resolution=1024x7682026.01 | 56.55 | — | — | — | — | — | — | — | 38.57 | — | — | — | |
| CMX-B4Year=2022, Backbone=Segformer-B4, Params/M=140, Real-Time=false, Multi-Scale inference (MS)=true, Input Resolution=480x6402023.09 | 56.3 | — | — | — | — | — | — | — | — | — | — | — | |
| DVEFormerBackbone=SwinV2-T-128, Extra Training Data=true, Resolution=H/4 x W/4, Evaluation Protocol=Linear probing2026.01 | 56.25 | — | — | — | — | — | — | — | 77 | — | — | — | |
| Multi-MAEYear=2022, Backbone=Vit-B, Input Resolution=480x6402023.09 | 56 | — | — | — | — | — | — | — | — | — | — | — | |
| MultiMAEBackbone=ViT-B, Extra Training Data=true2026.01 | 56 | — | — | — | — | — | — | — | — | — | — | — | |
| CMXBackbone=2x MiT-B42026.01 | 56 | — | — | — | — | — | — | — | 18.76 | — | — | — | |
| DFormerBackbone=DFormer-B, Extra Training Data=true2026.01 | 55.6 | — | — | — | — | — | — | — | 35.11 | — | — | — | |
| AsymFormerYear=2024, Backbone=B0+T, Params/M=33, Real-Time=false, Multi-Scale inference (MS)=true, Input Resolution=480x6402023.09 | 55.3 | — | — | — | — | — | — | — | — | — | — | — | |
| CMX-B2Year=2022, Backbone=Segformer-B2, Params/M=67, Real-Time=false, Multi-Scale inference (MS)=true, Input Resolution=480x6402023.09 | 54.4 | — | — | — | — | — | — | — | — | — | — | — | |
| Token-FusionYear=2022, Backbone=Token-Fusion(S), Input Resolution=480x6402023.09 | 54.2 | — | — | — | — | — | — | — | — | — | — | — | |
| CMX-B2Year=2022, Backbone=Segformer-B2, Params/M=67, Real-Time=false, Input Resolution=480x6402023.09 | 54.1 | — | — | — | — | — | — | — | 24 | 36.9 | — | — | |
| AsymFormerYear=2024, Backbone=B0+T, Params/M=33, Real-Time=true, Input Resolution=480x6402023.09 | 54.1 | — | — | — | — | — | — | — | 65 | 100 | — | — | |
| AsymFormerYear=2024, Backbone=B0+T, Params/M=33, Real-Time=true, Precision=FP16, Input Resolution=480x6402023.09 | 54.1 | — | — | — | — | — | — | — | 79 | 121.5 | — | — | |
| CMXBackbone=2x MiT-B22026.01 | 54.1 | — | — | — | — | — | — | — | 33.37 | — | — | — | |
| OmnivoreBackbone=Swin-B, Extra Training Data=true2026.01 | 54 | — | — | — | — | — | — | — | — | — | — | — | |
| PGDENetYear=2022, Backbone=Res34, Params/M=101, Real-Time=true, Input Resolution=480x6402023.09 | 53.7 | — | — | — | — | — | — | — | 32 | 49.2 | — | — | |
| ForkMerge2023.01 | 53.67 | 75.64 | — | — | — | — | — | 4.03 | — | — | — | — | |
| ForkMerge2023.01 | 53.67 | 75.64 | — | — | — | — | — | — | — | — | — | — | |
| FRNetYear=2022, Backbone=Res34, Params/M=86, Real-Time=true, Input Resolution=480x6402023.09 | 53.6 | — | — | — | — | — | — | — | 39 | 60 | — | — | |
| DFormer-SYear=2023, Backbone=DFormer-S, Params/M=18.7, Real-Time=true, Input Resolution=480x6402023.09 | 53.6 | — | — | — | — | — | — | — | 50 | 76.9 | — | — | |
| DFormerBackbone=DFormer-S, Extra Training Data=true2026.01 | 53.6 | — | — | — | — | — | — | — | 55.03 | — | — | — | |
| RLW2023.01 | 52.88 | 74.99 | — | — | — | — | — | 0.66 | — | — | — | — | |
| GradVac2023.01 | 52.84 | 74.77 | — | — | — | — | — | 0.75 | — | — | — | — | |
| ARML2023.01 | 52.73 | 74.85 | — | — | — | — | — | 0.37 | — | — | — | — | |
| ARML2023.01 | 52.73 | 74.85 | — | — | — | — | — | — | — | — | — | — | |
| GCS2023.01 | 52.67 | 74.59 | — | — | — | — | — | 0.09 | — | — | — | — | |
| GCS2023.01 | 52.67 | 74.59 | — | — | — | — | — | — | — | — | — | — | |
| MoDo2023.05 | 52.64 | 74.67 | — | — | — | — | — | — | — | — | — | — | |
| UW2023.01 | 52.51 | 74.72 | — | — | — | — | — | 0.63 | — | — | — | — | |
| UW2023.01 | 52.51 | 74.72 | — | — | — | — | — | — | — | — | — | — | |
| Auto-λ2023.01 | 52.4 | 74.62 | — | — | — | — | — | 1.17 | — | — | — | — | |
| Auto-λ2023.01 | 52.4 | 74.62 | — | — | — | — | — | — | — | — | — | — | |
| GradNorm2023.01 | 52.25 | 74.54 | — | — | — | — | — | 0.56 | — | — | — | — | |
| GradNorm2023.01 | 52.25 | 74.54 | — | — | — | — | — | — | — | — | — | — | |
| IMTL2023.01 | 52.24 | 74.73 | — | — | — | — | — | 2.1 | — | — | — | — | |
| IMTL2023.01 | 52.24 | 74.73 | — | — | — | — | — | — | — | — | — | — | |
| Warp-Refine Propagationlabels=warp-refine2021.09 | 52.2 | 77.6 | — | 66 | — | — | — | — | — | — | — | — | |
| EW2023.01 | 52.13 | 74.51 | — | — | — | — | — | 0.3 | — | — | — | — | |
| EW2023.01 | 52.13 | 74.51 | — | — | — | — | — | — | — | — | — | — | |
| DWA2023.01 | 52.1 | 74.45 | — | — | — | — | — | 0.07 | — | — | — | — | |
| DWA2023.01 | 52.1 | 74.45 | — | — | — | — | — | — | — | — | — | — | |
| Post-train2023.01 | 52.08 | 74.86 | — | — | — | — | — | 1.49 | — | — | — | — | |
| Post-train2023.01 | 52.08 | 74.86 | — | — | — | — | — | — | — | — | — | — | |
| OL_AUX2023.01 | 52.07 | 74.28 | — | — | — | — | — | 0.17 | — | — | — | — | |
| OL_AUX2023.01 | 52.07 | 74.28 | — | — | — | — | — | — | — | — | — | — | |
| CAGrad2023.01 | 52.04 | 74.25 | — | — | — | — | — | 1.41 | — | — | — | — | |
| CAGrad2023.01 | 52.04 | 74.25 | — | — | — | — | — | — | — | — | — | — | |
| Static2023.05 | 52.02 | 74.21 | — | — | — | — | — | — | — | — | — | — | |
| TeacherData=NYUv2, FLOPs=41G2021.10 | 51.9 | — | — | — | — | — | — | — | — | — | — | — | |
| TeacherTraining Data Scale=NYUv2 (original), Backbone=ResNet50, Architecture=DeepLabV32025.12 | 51.9 | — | — | — | — | — | — | — | — | — | — | — | |
| TeacherRequired data=Original data, FLOPS=41.0G, #params=24M2025.12 | 51.9 | — | — | — | — | — | — | — | — | — | — | — | |
| PCGrad2023.01 | 51.77 | 74.72 | — | — | — | — | — | 0.22 | — | — | — | — | |
| PCGrad2023.01 | 51.77 | 74.72 | — | — | — | — | — | — | — | — | — | — | |
| NashMTL2023.01 | 51.73 | 74.1 | — | — | — | — | — | 1.11 | — | — | — | — | |
| NashMTL2023.01 | 51.73 | 74.1 | — | — | — | — | — | — | — | — | — | — | |
| ESANetYear=2022, Backbone=Res34-Nbt1D, Params/M=34, Real-Time=true, Input Resolution=480x6402023.09 | 51.6 | — | — | — | — | — | — | — | 32 | 49.2 | — | — | |
| ESANet (pre. SceneNet)Backbone=2x ResNet-34 NBt1D, Pre-training=SceneNet, FPS=29.72020.11 | 51.58 | — | — | — | — | — | — | — | — | — | — | — | |
| STL2023.01 | 51.42 | 74.14 | — | — | — | — | — | — | — | — | — | — | |
| STL2023.01 | 51.42 | 74.14 | — | — | — | — | — | — | — | — | — | — | |
| CANetBackbone=ResNet-101, Multi-scale=true2022.06 | 51.2 | 76.6 | — | 63.8 | — | — | — | — | — | — | — | — | |
| Z-ACNBackbone=ResNet-101, Multi-scale=false2022.06 | 51.2 | 77 | 64.3 | 64.3 | — | — | — | — | — | — | — | — | |
| EMSANetBackbone=2x ResNet34-NBt1D2026.01 | 51.15 | — | — | — | — | — | — | — | 78.19 | — | — | — | |
| SGNetBackbone=ResNet-101, Multi-scale=true2022.06 | 51.1 | 76.8 | — | 63.1 | — | — | — | — | — | — | — | — | |
| EMSAFormerBackbone=SwinV2-T-128-Multi-Aug, Task(s)=Sem + Sce + Ins + Or, Semantic Decoder=EMSANet, Throughput (50W)=36.5 FPS, Throughput (30W)=25.6 FPS2023.06 | 51.06 | — | — | — | — | — | — | — | — | — | — | — | |
| EMSANetBackbone=2x ResNet34-NBt1D, Task(s)=Sem + Sce + Ins + Or, Throughput (50W)=70.5 FPS, Throughput (30W)=49.9 FPS2023.06 | 50.97 | — | — | — | — | — | — | — | — | — | — | — | |
| MalleableBackbone=ResNet-101, Multi-scale=true2022.06 | 50.9 | 76.9 | — | — | — | — | — | — | — | — | — | — | |
| EMSANetBackbone=2x ResNet101, Task(s)=Sem + Sce + Ins + Or, Throughput (50W)=42.9 FPS, Throughput (30W)=30.1 FPS2023.06 | 50.83 | — | — | — | — | — | — | — | — | — | — | — | |
| MGDA2023.01 | 50.79 | 73.81 | — | — | — | — | — | 1.44 | — | — | — | — | |
| MGDA2023.01 | 50.79 | 73.81 | — | — | — | — | — | — | — | — | — | — | |
| HaarNetExtra Parameters=+13.1M2023.10 | 50.7 | 77 | — | — | — | — | 39.2 | — | — | — | — | — | |
| DFSSRequired data=Unlabeled data, FLOPS=5.54G, #params=3.4M, sample size (epsilon)=20K2025.12 | 50.6 | — | — | — | — | — | — | — | — | — | — | — | |
| ESANet-R50Backbone=2x ResNet-50, FPS=22.62020.11 | 50.53 | — | — | — | — | — | — | — | — | — | — | — | |
| EMSAFormer (SwinV2-T-128-Multi-Aug single-task)Backbone=SwinV2-T-128-Multi-Aug, Task(s)=Semantic Segmentation (Sem), Throughput (50W)=47.1 FPS, Throughput (30W)=30.5 FPS2023.06 | 50.53 | — | — | — | — | — | — | — | — | — | — | — | |
| SA-GateBackbone=2x ResNet-50, FPS=11.92020.11 | 50.4 | — | — | — | — | — | — | — | — | — | — | — | |
| PAP-Net2021.09 | 50.4 | 76.2 | — | 62.5 | — | — | — | — | — | — | — | — | |
| SAGateExtra Parameters=+21.8M2023.10 | 50.4 | 76.8 | — | — | — | — | 38.1 | — | — | — | — | — | |
| DVEFormerBackbone=SwinV2-T-128, Extra Training Data=true, Evaluation Protocol=Visual mean2026.01 | 50.31 | — | — | — | — | — | — | — | 26.3 | — | — | — | |
| ESANet-R34-NBt1DBackbone=2x ResNet-34 NBt1D, FPS=29.72020.11 | 50.3 | — | — | — | — | — | — | — | — | — | — | — | |
| EMSAFormerBackbone=SwinV2-T-128-Multi-Aug, Task(s)=Sem(SegFormer) + Sce + Ins + Or, Semantic Decoder=SegFormer, Throughput (50W)=39.1 FPS, Throughput (30W)=27.3 FPS2023.06 | 50.23 | — | — | — | — | — | — | — | — | — | — | — | |
| EMSAFormerBackbone=SwinV2-T-1282026.01 | 50.23 | — | — | — | — | — | — | — | 62.63 | — | — | — | |
| PAD-Net2021.09 | 50.2 | 75.2 | — | 62.3 | — | — | — | — | — | — | — | — | |
| SA-GateYear=2021, Backbone=Res50, Params/M=65, Real-Time=true, Input Resolution=480x6402023.09 | 50.2 | — | — | — | — | — | — | — | 35 | 53.8 | — | — | |
| ShapeConvBackbone=ResNext-1012026.01 | 50.2 | — | — | — | — | — | — | — | — | — | — | — | |
| RDFNetModality=RGB-D, multi-scale evaluation=true2019.12 | 50.1 | 76 | — | 62.8 | — | — | — | — | — | — | — | — | |
| RDFNetBackbone=2x ResNet-152, Test-time augmentation=true, FPS=5.82020.11 | 50.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RDFNetBackbone=ResNet-152, Multi-scale=true2022.06 | 50.1 | 76 | — | 62.8 | — | — | — | — | — | — | — | — | |
| DVEFormerBackbone=SwinV2-T-128, Extra Training Data=true, Resolution=H/4 x W/4, Evaluation Protocol=Visual mean2026.01 | 50.02 | — | — | — | — | — | — | — | 77 | — | — | — | |
| IdempotentBackbone=2x ResNet-101, Inference complexity=∇ expected to be slower due to complex backbone, FPS=N/A2020.11 | 49.9 | — | — | — | — | — | — | — | — | — | — | — | |
| OursMode=RGB-D, Backbone=FasterNet-M2026.03 | 49.82 | — | — | — | — | — | — | — | — | — | — | — | |
| DFSSRequired data=Unlabeled data, FLOPS=5.54G, #params=3.4M, sample size (epsilon)=15K2025.12 | 49.8 | — | — | — | — | — | — | — | — | — | — | — | |
| EMSAFormerMode=RGB-D, Backbone=Swin v22026.03 | 49.76 | — | — | — | — | — | — | — | — | — | — | — | |
| OmnivoreBackbone=Swin-T, Extra Training Data=true2026.01 | 49.7 | — | — | — | — | — | — | — | — | — | — | — | |
| Malleable 2.5DMode=RGB-D, Backbone=ResNet502026.03 | 49.7 | — | — | — | — | — | — | — | — | — | — | — | |
| MMANetMode=RGB-D, Backbone=R34-NBt1D2026.03 | 49.62 | — | — | — | — | — | — | — | — | — | — | — |