Video Panoptic Segmentation on Cityscapes-VPS (val)
70.8VPQVideo K-Net
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Video K-NetBackbone=Swin-base + RFP, k=02022.04 | 70.8 | 63.2 | 76.3 | — | — | — | — | — | |
| ViP-DeepLabk=12020.12 | 70.4 | 63.2 | 75.7 | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41 + RFP + AutoAug, k=02022.04 | 69.2 | — | — | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base, k=02022.04 | 69.2 | 63.6 | 73.3 | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41, k=02022.04 | 68.2 | — | — | — | — | — | — | — | |
| Video K-NetBackbone=ResNet50, k=02022.04 | 65.6 | 57.4 | 71.5 | — | — | — | — | — | |
| PanopticFCN + OursTemporal window size=k = 0, FPS=5.12022.07 | 65.6 | 60 | 69.7 | — | — | — | — | — | |
| VPSNetk=12020.12 | 65 | 59 | 69.4 | — | — | — | — | — | |
| VPSNetBackbone=ResNet50, k=02022.04 | 65 | 59 | 69.4 | — | — | — | — | — | |
| VPSNet-FuseTrack + OursTemporal window size=k = 0, FPS=1.12022.07 | 65 | 59 | 69.4 | — | — | — | — | — | |
| SiamTrack-OursTemporal window size (k)=0, FPS=4.52021.06 | 64.6 | 58.3 | 69.1 | — | — | — | — | — | |
| SiamTrackBackbone=ResNet50, k=02022.04 | 64.6 | 58.3 | 69.1 | — | — | — | — | — | |
| SiamTrackTemporal window size=k = 0, FPS=4.52022.07 | 64.6 | 58.3 | 69.1 | — | — | — | — | — | |
| FuseTrackTemporal window size (k)=0, FPS=1.32021.06 | 64.5 | 58.1 | 69.1 | — | — | — | — | — | |
| VPSNet-FuseTrackTemporal window size=k = 0, FPS=1.32022.07 | 64.5 | 58.1 | 69.1 | — | — | — | — | — | |
| Slot-VPSDenseContext=true, AutoAug=true, RFP=true, SSL=false, TTA=false, Backbone=Swin-L-FPN, Image model M-adds (B)=19172021.12 | 63.7 | — | — | — | — | — | — | — | |
| ViP-DeepLabk=22020.12 | 63.6 | 50.7 | 73 | — | — | — | — | — | |
| SiamTrack-BaseTemporal window size (k)=0, FPS=4.52021.06 | 63.2 | 57 | 67.7 | — | — | — | — | — | |
| ViP-DeepLabk=average2020.12 | 63.1 | 49.5 | 73 | — | — | — | — | — | |
| TrackTemporal window size (k)=0, FPS=4.52021.06 | 63.1 | 56.4 | 68 | — | — | — | — | — | |
| ViP-DeeplabDenseContext=true, AutoAug=true, RFP=true, SSL=true, TTA=true, Backbone=WR-41, Image model M-adds (B)=32462021.12 | 63.1 | — | — | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base + RFP, k=52022.04 | 63.1 | 49.3 | 73.2 | — | — | — | — | — | |
| VPSNet-TrackTemporal window size=k = 0, FPS=4.52022.07 | 63.1 | 56.4 | 68 | — | — | — | — | — | |
| VIP-DeeplabShared Model=false2023.01 | 63.1 | 49.5 | 73 | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41 + RFP + AutoAug, k=52022.04 | 62.3 | — | — | — | — | — | — | — | |
| PolyphonicFormerBackbone=Swin-b, Test-time augmentation=false2021.12 | 62.3 | — | — | — | 70.8 | 63.1 | 59.5 | 56.8 | |
| Slot-VPSDenseContext=true, AutoAug=true, RFP=false, SSL=false, TTA=false, Backbone=Swin-L-FPN, Image model M-adds (B)=19172021.12 | 62.2 | — | — | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base + RFP, k=Average2022.04 | 62.2 | 49.8 | 71.8 | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base, k=52022.04 | 62 | 51.1 | 70 | — | — | — | — | — | |
| ViP-DeeplabDenseContext=true, AutoAug=true, RFP=true, SSL=false, TTA=true, Backbone=WR-41, Image model M-adds (B)=32462021.12 | 61.9 | — | — | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41 + RFP + AutoAug, k=Average2022.04 | 61.9 | — | — | — | — | — | — | — | |
| ViP-DeeplabBackbone=WR-41, Test-time augmentation=false2021.12 | 61.9 | — | — | — | 69.2 | 62.3 | 59.2 | 57 | |
| ViP-DeeplabBackbone=WideResNet41, k=52022.04 | 61.3 | — | — | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base, k=Average2022.04 | 61.2 | 49.6 | 69.5 | — | — | — | — | — | |
| ViP-DeeplabDenseContext=true, AutoAug=true, RFP=true, SSL=false, TTA=false, Backbone=WR-41, Image model M-adds (B)=32462021.12 | 60.9 | — | — | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41, k=Average2022.04 | 60.9 | — | — | — | — | — | — | — | |
| ViP-DeepLabk=32020.12 | 60.1 | 44 | 71.9 | — | — | — | — | — | |
| Slot-VPSBackbone=ResNet50-FPN2021.12 | 59.7 | 47.5 | 68.6 | 4.6 | — | — | — | — | |
| Video K-NetBackbone=Swin-base + RFP, k=102022.04 | 59.5 | 43.4 | 72 | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41 + RFP + AutoAug, k=102022.04 | 59.2 | — | — | — | — | — | — | — | |
| TarViSShared Model=true, Backbone=Swin-L2023.01 | 58.9 | 43.7 | 69.9 | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base, k=102022.04 | 58.4 | 44.7 | 68.3 | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41, k=102022.04 | 58.2 | — | — | — | — | — | — | — | |
| PolyphonicFormerBackbone=ResNet50, Test-time augmentation=false2021.12 | 58.2 | — | — | — | 65.4 | 58.6 | 55.4 | 53.3 | |
| ViP-DeepLabk=42020.12 | 58.1 | 40.2 | 71.2 | — | — | — | — | — | |
| TarViSShared Model=true, Backbone=Swin-T2023.01 | 58 | 42.9 | 69 | — | — | — | — | — | |
| Video K-NetBackbone=ResNet50, k=Average2022.04 | 57.8 | 45 | 66.9 | — | — | — | — | — | |
| PanopticFCN + OursTemporal window size=k = 5, FPS=5.12022.07 | 57.8 | 45.7 | 66.6 | — | — | — | — | — | |
| Video K-NetBackbone=ResNet50, k=52022.04 | 57.7 | 43.4 | 68.2 | — | — | — | — | — | |
| VPSNetk=22020.12 | 57.6 | 45.1 | 66.7 | — | — | — | — | — | |
| SiamTrack-OursTemporal window size (k)=5, FPS=4.52021.06 | 57.6 | 45.6 | 66.6 | — | — | — | — | — | |
| VPSNetBackbone=ResNet50, k=52022.04 | 57.6 | 45.1 | 66.7 | — | — | — | — | — | |
| SiamTrackBackbone=ResNet50, k=52022.04 | 57.6 | 45.6 | 66.6 | — | — | — | — | — | |
| SiamTrackTemporal window size=k = 5, FPS=4.52022.07 | 57.6 | 45.6 | 66.6 | — | — | — | — | — | |
| VPSNet-FuseTrack + OursTemporal window size=k = 5, FPS=1.12022.07 | 57.6 | 45 | 66.7 | — | — | — | — | — | |
| VPSNetk=average2020.12 | 57.5 | 44.8 | 66.7 | — | — | — | — | — | |
| VPSNetBackbone=ResNet50, k=Average2022.04 | 57.5 | 44.8 | 66.7 | — | — | — | — | — | |
| VPSNetTest-time augmentation=false2021.12 | 57.5 | — | — | — | 65 | 57.6 | 54.4 | 52.8 | |
| PanopticFCN + OursTemporal window size=Average (k in {0,5,10,15}), FPS=5.12022.07 | 57.5 | 45.1 | 66.4 | — | — | — | — | — | |
| VPSNet-FuseTrack + OursTemporal window size=Average (k in {0,5,10,15}), FPS=1.12022.07 | 57.5 | 44.7 | 66.7 | — | — | — | — | — | |
| FuseTrackTemporal window size (k)=5, FPS=1.32021.06 | 57.4 | 45.2 | 66.4 | — | — | — | — | — | |
| VPSNet-FuseTrackTemporal window size=k = 5, FPS=1.32022.07 | 57.4 | 45.2 | 66.4 | — | — | — | — | — | |
| SiamTrack-OursTemporal window size (k)=average, FPS=4.52021.06 | 57.3 | 44.7 | 66.4 | — | — | — | — | — | |
| SiamTrackBackbone=ResNet50-FPN2021.12 | 57.3 | 44.7 | 66.4 | 4.6 | — | — | — | — | |
| SiamTrackBackbone=ResNet50, k=Average2022.04 | 57.3 | 44.7 | 65 | — | — | — | — | — | |
| SiamTrackTest-time augmentation=false2021.12 | 57.3 | — | — | — | 64.6 | 57.6 | 54.2 | 52.7 | |
| SiamTrackTemporal window size=Average (k in {0,5,10,15}), FPS=4.52022.07 | 57.3 | 44.7 | 66.4 | — | — | — | — | — | |
| VPSNet-SiamTrackShared Model=false2023.01 | 57.3 | 44.7 | 66.4 | — | — | — | — | — | |
| FuseTrackTemporal window size (k)=average, FPS=1.32021.06 | 57.2 | 44.7 | 66.6 | — | — | — | — | — | |
| VPSNet-FuseTrackTemporal window size=Average (k in {0,5,10,15}), FPS=1.32022.07 | 57.2 | 44.7 | 66.6 | — | — | — | — | — | |
| VPSNetBackbone=ResNet50-FPN2021.12 | 57 | 44.7 | 66 | 1.3 | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41 + RFP + AutoAug, k=152022.04 | 57 | — | — | — | — | — | — | — | |
| VPSNetShared Model=false2023.01 | 57 | 44.7 | 66 | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base + RFP, k=152022.04 | 56.8 | 37 | 71.1 | — | — | — | — | — | |
| SiamTrack-BaseTemporal window size (k)=5, FPS=4.52021.06 | 56.2 | 42.6 | 66.3 | — | — | — | — | — | |
| ViP-DeeplabBackbone=WideResNet41, k=152022.04 | 56.2 | — | — | — | — | — | — | — | |
| TrackTemporal window size (k)=5, FPS=4.52021.06 | 56.1 | 44.1 | 64.9 | — | — | — | — | — | |
| VPSNet-TrackTemporal window size=k = 5, FPS=4.52022.07 | 56.1 | 44.1 | 64.9 | — | — | — | — | — | |
| SiamTrack-BaseTemporal window size (k)=average, FPS=4.52021.06 | 56 | 42.9 | 65.8 | — | — | — | — | — | |
| TrackTemporal window size (k)=average, FPS=4.52021.06 | 55.9 | 43.7 | 64.8 | — | — | — | — | — | |
| VPSNet-TrackTemporal window size=Average (k in {0,5,10,15}), FPS=4.52022.07 | 55.9 | 43.7 | 64.8 | — | — | — | — | — | |
| TrackShared Model=false2023.01 | 55.9 | 43.7 | 64.8 | — | — | — | — | — | |
| Video K-NetBackbone=Swin-base, k=152022.04 | 55.8 | 39.8 | 67.5 | — | — | — | — | — | |
| VPSNetk=32020.12 | 54.4 | 39.2 | 65.6 | — | — | — | — | — | |
| VPSNetBackbone=ResNet50, k=102022.04 | 54.4 | 39.2 | 65.6 | — | — | — | — | — | |
| VPSNet-FuseTrack + OursTemporal window size=k = 10, FPS=1.12022.07 | 54.4 | 39.1 | 65.6 | — | — | — | — | — | |
| PanopticFCN + OursTemporal window size=k = 10, FPS=5.12022.07 | 54.3 | 39.5 | 65.1 | — | — | — | — | — | |
| SiamTrack-OursTemporal window size (k)=10, FPS=4.52021.06 | 54.2 | 39.2 | 65.2 | — | — | — | — | — | |
| SiamTrackBackbone=ResNet50, k=102022.04 | 54.2 | 39.2 | 65.2 | — | — | — | — | — | |
| Video K-NetBackbone=ResNet50, k=102022.04 | 54.2 | 36.5 | 67.1 | — | — | — | — | — | |
| SiamTrackTemporal window size=k = 10, FPS=4.52022.07 | 54.2 | 39.2 | 65.2 | — | — | — | — | — | |
| FuseTrackTemporal window size (k)=10, FPS=1.32021.06 | 54.1 | 39.5 | 64.7 | — | — | — | — | — | |
| VPSNet-FuseTrackTemporal window size=k = 10, FPS=1.32022.07 | 54.1 | 39.5 | 64.7 | — | — | — | — | — | |
| TarViSShared Model=true, Backbone=R-502023.01 | 53.3 | 35.9 | 66 | — | — | — | — | — | |
| SiamTrack-BaseTemporal window size (k)=10, FPS=4.52021.06 | 53.2 | 38.3 | 65.2 | — | — | — | — | — | |
| TrackTemporal window size (k)=10, FPS=4.52021.06 | 53.1 | 39 | 63.4 | — | — | — | — | — | |
| VPSNet-TrackTemporal window size=k = 10, FPS=4.52022.07 | 53.1 | 39 | 63.4 | — | — | — | — | — | |
| VPSNetk=42020.12 | 52.8 | 35.8 | 65.3 | — | — | — | — | — | |
| VPSNetBackbone=ResNet50, k=152022.04 | 52.8 | 35.8 | 65.3 | — | — | — | — | — | |
| VPSNet-FuseTrack + OursTemporal window size=k = 15, FPS=1.12022.07 | 52.8 | 35.8 | 65.2 | — | — | — | — | — |