Referring Video Object Segmentation on JHMDB-Sentences (with P@X metrics)
74.4Overall IoUVD-IT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| VD-ITBackbone=Video Diffusion2024.03 | 74.4 | — | — | — | — | — | 73.4 | 46.5 | |
| HTRBackbone=VideoSwin-B2024.03 | 73.9 | 97.1 | 92 | 72.1 | 2,370 | 0.3 | 72.6 | 44.9 | |
| SgMgBackbone=Video-Swin-B2024.03 | 73.7 | — | — | — | — | — | 72.5 | 45 | |
| SOCBackbone=Video-Swin-B, Image Pretrain=true2023.05 | 73.6 | 96.9 | 91.4 | 71.1 | 2,130 | 0.1 | 72.3 | 44.6 | |
| OnlineReferBackbone=Video-Swin-B, Online/Offline=semi-online, Window size (w)=52023.07 | 73.5 | 96.1 | 90.4 | 71 | 2,190 | 0.2 | 71.9 | — | |
| OnlineReferBackbone=VideoSwin-B2024.03 | 73.5 | 96.1 | 90.4 | 71 | 2,190 | 0.2 | 71.9 | — | |
| ReferFormerBackbone=Video-Swin-B, Window size (w)=52022.01 | 73 | 96.2 | 90.2 | 70.2 | 2,100 | 0.3 | 71.8 | 43.7 | |
| ReferFormerBackbone=Video-Swin-B, Image Pretrain=true2023.05 | 73 | 96.2 | 90.2 | 70.2 | 2,100 | 0.3 | 71.8 | 43.7 | |
| ReferFormerBackbone=Video-Swin-B, Online/Offline=offline, Window size (w)=52023.07 | 73 | 96.2 | 90.2 | 70.2 | 2,100 | 0.3 | 71.8 | — | |
| ReferFormerBackbone=Video-Swin-B2024.03 | 73 | — | — | — | — | — | 71.8 | 43.7 | |
| ReferFormerBackbone=VideoSwin-B2024.03 | 73 | 96.2 | 90.2 | 70.2 | 2,100 | 0.3 | 71.8 | 43.7 | |
| ReferFormerBackbone=Video-Swin-S, Window size (w)=52022.01 | 72.8 | 95.8 | 90.1 | 68.7 | 2,030 | 0.2 | 71.5 | 42.4 | |
| SgMgBackbone=Video-Swin-T2024.03 | 72.8 | — | — | — | — | — | 71.7 | 44.4 | |
| SOCBackbone=Video-Swin-T, Image Pretrain=true2023.05 | 72.7 | 96.3 | 88.7 | 67.2 | 1,960 | 0.1 | 71.6 | 42.7 | |
| ReferFormerBackbone=Video-Swin-T, Window size (w)=52022.01 | 71.9 | 95.8 | 89.3 | 66.8 | 1,890 | 0.2 | 71 | 42.2 | |
| ReferFormerBackbone=Video-Swin-T, Image Pretrain=true2023.05 | 71.9 | 95.8 | 89.3 | 66.8 | 1,890 | 0.2 | 71 | 42.2 | |
| ReferFormerBackbone=Video-Swin-T, Online/Offline=offline, Window size (w)=52023.07 | 71.9 | 95.8 | 89.3 | 66.8 | 1,890 | 0.2 | 71 | — | |
| ReferFormerBackbone=Video-Swin-T2024.03 | 71.9 | — | — | — | — | — | 71 | 42.2 | |
| SOCBackbone=Video-Swin-T, Image Pretrain=false2023.05 | 70.7 | 94.7 | 86.4 | 62.7 | 1,790 | 0.1 | 70.1 | 39.7 | |
| MTTRBackbone=Video-Swin-T, Window size (w)=102022.01 | 70.1 | 93.9 | 85.2 | 61.6 | 1,660 | 0.1 | 69.8 | 39.2 | |
| MTTRBackbone=Video-Swin-T, Image Pretrain=false2023.05 | 70.1 | 93.9 | 85.2 | 61.6 | 1,660 | 0.1 | 69.8 | 39.2 | |
| MTTRBackbone=Video-Swin-T, Online/Offline=offline, Window size (w)=102023.07 | 70.1 | 93.9 | 85.2 | 61.6 | 1,660 | 0.1 | 69.8 | — | |
| MTTRBackbone=Video-Swin-T2024.03 | 70.1 | — | — | — | — | — | 69.8 | 39.2 | |
| MTTRBackbone=VideoSwin-T2024.03 | 70.1 | 93.9 | 85.2 | 61.6 | 1,660 | 0.1 | 69.8 | 39.2 | |
| ReferFormerBackbone=Video-Swin-T, Window size (w)=6, Training protocol=trained from scratch2022.01 | 70 | 93.3 | 84.2 | 61.4 | 1,640 | 0.3 | 69.3 | 39.1 | |
| ReferFormerBackbone=Video-Swin-T, Image Pretrain=false2023.05 | 70 | 93.3 | 84.2 | 61.4 | 1,640 | 0.3 | 69.3 | 39.1 | |
| MTTRBackbone=Video-Swin-T, Window size (w)=82022.01 | 67.4 | 91 | 81.5 | 57 | 1,440 | 0.1 | 67.9 | 36.6 | |
| ClawCraneNetBackbone=ResNet-50/1012022.01 | 64.4 | 88 | 79.6 | 56.6 | 1,470 | 0.2 | 65.6 | — | |
| ClawCraneNetBackbone=ResNet-50/101, Image Pretrain=false2023.05 | 64.4 | 88 | 79.6 | 56.6 | 1,470 | 0.2 | 65.6 | — | |
| ClawCraneNetBackbone=ResNet-50/101, Online/Offline=offline2023.07 | 64.4 | 88 | 79.6 | 56.6 | 1,470 | 0.2 | 65.6 | — | |
| ClawCraneNetBackbone=ResNet-50/1012024.03 | 64.4 | — | — | — | — | — | 65.6 | — | |
| ClawCraneNetBackbone=ResNet-50/1012024.03 | 64.4 | 88 | 79.6 | 56.6 | 1,470 | 0.2 | 65.6 | — | |
| CMSA + CFSABackbone=ResNet-1012022.01 | 62.8 | 76.4 | 62.5 | 38.9 | 900 | 0.1 | 58.1 | — | |
| CMSA + CFSABackbone=ResNet-101, Image Pretrain=false2023.05 | 62.8 | 76.4 | 62.5 | 38.9 | 900 | 0.1 | 58.1 | — | |
| CMSA + CFSABackbone=ResNet-101, Online/Offline=offline2023.07 | 62.8 | 76.4 | 62.5 | 38.9 | 900 | 0.1 | 58.1 | — | |
| CMSA + CFSABackbone=ResNet-1012024.03 | 62.8 | 76.4 | 62.5 | 38.9 | 900 | 0.1 | 58.1 | — | |
| CMPC-VBackbone=I3D2022.01 | 61.6 | 81.3 | 65.7 | 37.1 | 700 | 0 | 61.7 | 34.2 | |
| CMPC-VBackbone=I3D, Image Pretrain=false2023.05 | 61.6 | 81.3 | 65.7 | 37.1 | 700 | 0 | 61.7 | 34.2 | |
| CMPC-VBackbone=I3D, Online/Offline=offline2023.07 | 61.6 | 81.3 | 65.7 | 37.1 | 700 | 0 | 61.7 | — | |
| CMPC-VBackbone=I3D2024.03 | 61.6 | — | — | — | — | — | 61.7 | 31.2 | |
| CMPC-VBackbone=I3D2024.03 | 61.6 | 81.3 | 65.7 | 37.1 | 700 | 0 | 61.7 | 34.2 | |
| ACANBackbone=I3D2022.01 | 57.6 | 75.6 | 56.4 | 28.7 | 340 | 0 | 58.4 | 28.9 | |
| ACANBackbone=I3D, Image Pretrain=false2023.05 | 57.6 | 75.6 | 56.4 | 28.7 | 340 | 0 | 58.4 | 28.9 | |
| ACANBackbone=I3D, Online/Offline=offline2023.07 | 57.6 | 75.6 | 56.4 | 28.7 | 340 | 0 | 58.4 | — | |
| ACANBackbone=I3D2024.03 | 57.6 | — | — | — | — | — | 58.4 | 28.9 | |
| ACANBackbone=I3D2024.03 | 57.6 | 75.6 | 56.4 | 28.7 | 340 | 0 | 58.4 | 28.9 | |
| Hu et al.Backbone=VGG-162022.01 | 54.6 | 63.3 | 35 | 8.5 | 20 | 0 | 52.8 | 17.8 | |
| Hu et al.Backbone=VGG-16, Image Pretrain=false2023.05 | 54.6 | 63.3 | 35 | 8.5 | 20 | 0 | 52.8 | 17.8 | |
| Hu et al.Backbone=VGG-16, Online/Offline=offline2023.07 | 54.6 | 63.3 | 35 | 8.5 | 20 | 0 | 52.8 | — | |
| Hu et al.Backbone=VGG-162024.03 | 54.6 | — | — | — | — | — | 52.8 | 17.8 | |
| Hu et al.Backbone=VGG-162024.03 | 54.6 | 63.3 | 35 | 8.5 | 20 | 0 | 52.8 | 17.8 | |
| Gavrilyuk et al.Backbone=I3D2022.01 | 54.1 | 69.9 | 46 | 17.3 | 140 | 0 | 54.2 | 23.3 | |
| Gavrilyuk et al.Backbone=I3D, Image Pretrain=false2023.05 | 54.1 | 69.9 | 46 | 17.3 | 140 | 0 | 54.2 | 23.3 | |
| Gavrilyuk et al.Backbone=I3D, Online/Offline=offline2023.07 | 54.1 | 69.9 | 46 | 17.3 | 140 | 0 | 54.2 | — | |
| Gavrilyuk et al.Backbone=I3D2024.03 | 54.1 | — | — | — | — | — | 54.2 | 23.3 | |
| Gavrilyuk et al.Backbone=I3D2024.03 | 54.1 | 69.9 | 46 | 17.3 | 140 | 0 | 54.2 | 23.3 |