Referring Video Object Segmentation on A2D Sentences (test)
81.1Overall IoUDsHmp
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DsHmpBackbone=Video-Swin-Base2024.04 | 81.1 | — | — | — | — | — | — | 72.9 | — | 59.8 | — | |
| SOCBackbone=Video-Swin-Base2024.04 | 80.7 | — | — | — | — | — | — | 72.5 | — | 57.3 | — | |
| SgMgBackbone=Video-Swin-B2023.07 | 79.9 | — | — | — | — | — | — | 72 | — | 58.5 | — | |
| SgMgBackbone=Video-Swin-Base2024.04 | 79.9 | — | — | — | — | — | — | 72 | — | 58.5 | — | |
| OnlineReferBackbone=Video-Swin-B, Online/Offline=semi-online, Window Size=w=52023.07 | 79.6 | 83.1 | 80.2 | 73.4 | 56.8 | 2,170 | — | 70.5 | — | — | — | |
| OnlineReferBackbone=Video-Swin-Base2024.04 | 79.6 | — | — | — | — | — | — | 70.5 | — | — | — | |
| HTMLBackbone=Video-Swin-Base2024.04 | 79.5 | — | — | — | — | — | — | 71.2 | — | 56.7 | — | |
| DsHmpBackbone=Video-Swin-Tiny2024.04 | 79 | — | — | — | — | — | — | 71.3 | — | 57.2 | — | |
| ReferFormerBackbone=Video-Swin-B, Window size (w)=5, Trained from scratch=false2022.01 | 78.6 | 83.1 | 80.4 | 74.1 | 57.9 | 2,120 | — | 70.3 | 55 | — | — | |
| ReferFormerBackbone=Video-Swin-B, Online/Offline=offline, Window Size=w=52023.07 | 78.6 | 83.1 | 80.4 | 74.1 | 57.9 | 2,120 | — | 70.3 | — | — | — | |
| ReferFormerBackbone=Video-Swin-B2023.07 | 78.6 | — | — | — | — | — | — | 70.3 | — | 55 | — | |
| ReferFormerBackbone=Video-Swin-Base2024.04 | 78.6 | — | — | — | — | — | — | 70.3 | — | 55 | — | |
| SOCBackbone=Video-Swin-Tiny2024.04 | 78.3 | — | — | — | — | — | — | 70.6 | — | 54.8 | — | |
| SgMgBackbone=Video-Swin-T2023.07 | 78 | — | — | — | — | — | — | 70.4 | — | 56.1 | — | |
| SgMgBackbone=Video-Swin-Tiny2024.04 | 78 | — | — | — | — | — | — | 70.4 | — | 56.1 | — | |
| ReferFormerBackbone=Video-Swin-S, Window size (w)=5, Trained from scratch=false2022.01 | 77.7 | 82.6 | 79.4 | 73.1 | 57.4 | 2,110 | — | 69.8 | 53.9 | — | — | |
| ReferFormerBackbone=Video-Swin-T, Window size (w)=5, Trained from scratch=false2022.01 | 77.6 | 82.8 | 79.2 | 72.3 | 55.3 | 1,930 | — | 69.6 | 52.8 | — | — | |
| ReferFormerBackbone=Video-Swin-T, Online/Offline=offline, Window Size=w=52023.07 | 77.6 | 82.8 | 79.2 | 72.3 | 55.3 | 1,930 | — | 69.6 | — | — | — | |
| ReferFormerBackbone=Video-Swin-T2023.07 | 77.6 | — | — | — | — | — | — | 69.6 | — | 52.8 | — | |
| ReferFormerBackbone=Video-Swin-Tiny2024.04 | 77.6 | — | — | — | — | — | — | 69.6 | — | 52.8 | — | |
| HTMLBackbone=Video-Swin-Tiny2024.04 | 77.6 | — | — | — | — | — | — | 69.2 | — | 53.4 | — | |
| ReferFormerBackbone=Video-Swin-T, Window size (w)=6, Trained from scratch=true2022.01 | 72.3 | 76 | 72.2 | 65.4 | 49.8 | 1,790 | — | 64.1 | 48.6 | — | — | |
| MTTRBackbone=Video-Swin-T, Window size (w)=102022.01 | 72 | 75.4 | 71.2 | 63.8 | 48.5 | 1,690 | — | 64 | 46.1 | — | — | |
| MTTRw=102021.11 | 72 | 75.4 | 71.2 | 63.8 | 48.5 | 16.9 | — | 64 | — | 46.1 | — | |
| MTTRBackbone=Video-Swin-T, Online/Offline=offline, Window Size=w=102023.07 | 72 | 75.4 | 71.2 | 63.8 | 48.5 | 1,690 | — | 64 | — | — | — | |
| MTTRBackbone=Video-Swin-T2023.07 | 72 | — | — | — | — | — | — | 64 | — | 46.1 | — | |
| MTTRBackbone=Video-Swin-Tiny2024.04 | 72 | — | — | — | — | — | — | 64 | — | 46.1 | — | |
| Ours2022.03 | 71.4 | 70.2 | 66.3 | 58.5 | 42.8 | 15.1 | 46.9 | 59.8 | — | — | — | |
| LBDTLBDT layers=42022.06 | 70.4 | 73 | 67.4 | 59 | 42.1 | 13.2 | 47.2 | 62.1 | — | — | — | |
| MTTRBackbone=Video-Swin-T, Window size (w)=82022.01 | 70.2 | 72.1 | 68.4 | 60.7 | 45.6 | 1,640 | — | 61.8 | 44.7 | — | — | |
| MTTRw=82021.11 | 70.2 | 72.1 | 68.4 | 60.7 | 45.6 | 16.4 | — | 61.8 | — | 44.7 | — | |
| LBDTLBDT layers=12022.06 | 70.1 | 71.1 | 66.1 | 57.8 | 41.6 | 12 | 46.1 | 61.2 | — | — | — | |
| MMVT+LGFF (B+M+T+L+A)additional optical flow input=true2022.04 | 67.3 | 64.5 | 59.7 | 52.3 | 37.5 | 13 | 41.9 | 55.8 | — | — | — | |
| RefVOS2021.11 | 67.2 | 57.8 | — | — | — | 9.3 | — | 49.7 | — | — | — | |
| OursInput=RGB2021.05 | 66.2 | 65.4 | 58.9 | 49.7 | 33.3 | 9.1 | 39.9 | 56.1 | — | — | — | |
| LGFS212022.03 | 66.2 | 65.4 | 58.9 | 49.7 | 33.3 | 9.1 | 39.9 | 56.1 | — | — | — | |
| CSTM2021.11 | 66.2 | 65.4 | 58.9 | 49.7 | 33.3 | 9.1 | — | 56.1 | — | 39.9 | — | |
| CSTMPub.=CVPR212022.06 | 66.2 | 65.4 | 58.9 | 49.7 | 33.3 | 9.1 | 39.9 | 56.1 | — | — | — | |
| CSTMVenue=CVPR2021, additional optical flow input=false2022.04 | 66.2 | 65.4 | 58.9 | 49.7 | 33.3 | 9.1 | 39.9 | 56.1 | — | — | — | |
| PRPEInput=RGB2021.05 | 66.1 | 63.4 | 57.9 | 48.3 | 32.2 | 8.3 | 38.8 | 52.9 | — | — | — | |
| PRPE2021.05 | 66.1 | 63.4 | 57.9 | 48.3 | 32.2 | 8.3 | 38.8 | 52.9 | — | — | — | |
| PRPE202022.03 | 66.1 | 63.4 | 57.9 | 48.3 | 32.2 | 8.3 | 38.8 | 52.9 | — | — | — | |
| PRPEPub.=IJCAI202022.06 | 66.1 | 63.4 | 57.9 | 48.3 | 32.2 | 8.3 | 38.8 | 52.9 | — | — | — | |
| Ning et al.Training=A2D Sentences exclusively2021.03 | 66.1 | 63.4 | 57.9 | 48.3 | 32.2 | 830 | 38.8 | 52.9 | — | — | 5.42 | |
| PRPEVenue=IJCAI2020, additional optical flow input=false2022.04 | 66.1 | 63.4 | 57.9 | 48.3 | 32.2 | 8.3 | 38.8 | 52.9 | — | — | — | |
| Ours-I3DBackbone=3D (I3D)2021.05 | 65.3 | 65.5 | 59.2 | 50.6 | 34.2 | 9.8 | 40.4 | 57.3 | — | — | — | |
| CMPC-VBackbone=I3D2022.01 | 65.3 | 65.5 | 59.2 | 50.6 | 34.2 | 980 | — | 57.3 | 40.4 | — | — | |
| CMPC-V212022.03 | 65.3 | 65.5 | 59.2 | 50.6 | 34.2 | 9.8 | 40.4 | 57.3 | — | — | — | |
| CMPC-VBackbone=I3D2021.11 | 65.3 | 65.5 | 59.2 | 50.6 | 34.2 | 9.8 | — | 57.3 | — | 40.4 | — | |
| CMPC-VPub.=TPAMI212022.06 | 65.3 | 65.5 | 59.2 | 50.6 | 34.2 | 9.8 | 40.4 | 57.3 | — | — | — | |
| CMPC-VBackbone=I3D, Online/Offline=offline2023.07 | 65.3 | 65.5 | 59.2 | 50.6 | 34.2 | 980 | — | 57.3 | — | — | — | |
| CMPC-VBackbone=I3D2023.07 | 65.3 | — | — | — | — | — | — | 57.3 | — | 40.4 | — | |
| Ours-R2DBackbone=2D (R2D)2021.05 | 64.9 | 59 | 52.7 | 43.4 | 28.4 | 6.8 | 35.1 | 51.5 | — | — | — | |
| CMPC2020.11 | 64.9 | 59 | 52.7 | 43.4 | 28.4 | 6.8 | 35.1 | 51.5 | — | — | — | |
| AAMN2020.11 | 63.4 | 68.9 | 64.2 | 54.5 | 32.4 | 3.4 | 41.2 | 55.6 | — | — | — | |
| ClawCraneNetBackbone=ResNet-50/1012022.01 | 63.1 | 70.4 | 67.7 | 61.7 | 48.9 | 1,710 | — | 59.9 | — | — | — | |
| Claw CraneNetBackbone=ResNet-50/101, Online/Offline=offline2023.07 | 63.1 | 70.4 | 67.7 | 61.7 | 48.9 | 1,710 | — | 59.9 | — | — | — | |
| ClawCraneNetVisual Backbone=ResNet-50, Text Encoder=bi-LSTM, Input Resolution=320x320, Training=A2D Sentences exclusively2021.03 | 63.1 | 70.4 | 67.7 | 61.7 | 48.9 | 1,710 | 49.4 | 59.9 | — | — | 9.27 | |
| BRINet2020.11 | 63 | 55.7 | 46.4 | 33 | 17.5 | 3.3 | 28.3 | 47.4 | — | — | — | |
| CMDYInput=RGB2021.05 | 62.3 | 60.7 | 52.5 | 40.5 | 23.5 | 4.5 | 33.3 | 53.1 | — | — | — | |
| CMDY202022.03 | 62.3 | 60.7 | 52.5 | 40.5 | 23.5 | 4.5 | 33.3 | 53.1 | — | — | — | |
| CMDyPub.=AAAI202022.06 | 62.3 | 60.7 | 52.5 | 40.5 | 23.5 | 4.5 | 33.3 | 53.1 | — | — | — | |
| Wang et al.Training=A2D Sentences exclusively2021.03 | 62.3 | 60.7 | 52.5 | 40.5 | 23.5 | 450 | 33.3 | 53.1 | — | — | 7.18 | |
| CMDYVenue=AAAI2020, additional optical flow input=false2022.04 | 62.3 | 60.7 | 52.5 | 40.5 | 23.5 | 4.5 | 33.3 | 53.1 | — | — | — | |
| CMSA + CFSABackbone=ResNet-1012022.01 | 61.8 | 48.7 | 43.1 | 35.8 | 23.1 | 520 | — | 43.2 | — | — | — | |
| CMSA-V212022.03 | 61.8 | 48.7 | 43.1 | 35.8 | 23.1 | 5.2 | — | 43.2 | — | — | — | |
| CMSA+CFSA2021.11 | 61.8 | 48.7 | 43.1 | 35.8 | 23.1 | 5.2 | — | 43.2 | — | — | — | |
| CMSAPub.=TPAMI212022.06 | 61.8 | 48.7 | 43.1 | 35.8 | 23.1 | 5.2 | — | 43.2 | — | — | — | |
| CMSA + CFSABackbone=ResNet-101, Online/Offline=offline2023.07 | 61.8 | 48.7 | 43.1 | 35.8 | 23.1 | 520 | — | 43.2 | — | — | — | |
| AAMN2021.11 | 61.7 | 68.1 | 62.9 | 52.3 | 29.6 | 2.9 | — | 55.2 | — | 39.6 | — | |
| ACGAInput=RGB2021.05 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 2 | 27.4 | 49 | — | — | — | |
| ACGANetModality=RGB2021.05 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 2 | 27.4 | 49 | — | — | — | |
| ACANBackbone=I3D2022.01 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 200 | — | 49 | 27.4 | — | — | |
| ACGA192022.03 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 2 | 27.4 | 49 | — | — | — | |
| ACGAPub.=ICCV192022.06 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 2 | 27.4 | 49 | — | — | — | |
| ACANBackbone=I3D, Online/Offline=offline2023.07 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 200 | — | 49 | — | — | — | |
| ACANBackbone=I3D2023.07 | 60.1 | — | — | — | — | — | — | 49 | — | 27.4 | — | |
| Wang et al.Training=A2D Sentences exclusively2021.03 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 200 | 27.4 | 49 | — | — | 8.64 | |
| ACGAVenue=ICCV2019, additional optical flow input=false2022.04 | 60.1 | 55.7 | 45.9 | 31.9 | 16 | 2 | 27.4 | 49 | — | — | — | |
| ClawCraneNetBackbone=ResNet-50/1012023.07 | 59.9 | — | — | — | — | — | — | — | — | 63.1 | — | |
| CMSA2020.11 | 59.2 | 46.7 | 38.5 | 27.9 | 13.6 | 1.7 | — | 40.5 | — | — | — | |
| Gavrilyuk et al.Training=A2D Sentences exclusively2021.03 | 57.4 | 53.8 | 43.7 | 31.8 | 17.1 | 210 | 26.9 | 48.1 | — | — | — | |
| VT-CapsuleInput=RGB2021.05 | 56.8 | 52.6 | 45 | 34.5 | 20.7 | 3.6 | 30.3 | 46 | — | — | — | |
| VT-Capsule202022.03 | 56.8 | 52.6 | 45 | 34.5 | 20.7 | 3.6 | 30.3 | 46 | — | — | — | |
| VT-CapsulePub.=CVPR202022.06 | 56.8 | 52.6 | 45 | 34.5 | 20.7 | 3.6 | 30.3 | 46 | — | — | — | |
| McIntosh et al.Training=A2D Sentences exclusively2021.03 | 56.8 | 52.6 | 45 | 34.5 | 20.7 | 360 | 30.3 | 46 | — | — | — | |
| VT-CapsuleVenue=CVPR2020, additional optical flow input=false2022.04 | 56.8 | 52.6 | 45 | 34.5 | 20.7 | 3.6 | 30.3 | 46 | — | — | — | |
| RRN2020.11 | 56.4 | 42 | 35.5 | 27.5 | 15.4 | 2.5 | 22.4 | 37.3 | — | — | — | |
| fully-convolutional modelInput modality=RGB + Flow2018.03 | 55.1 | 50 | 37.6 | 23.1 | 9.4 | 0.4 | 21.5 | 42.6 | — | — | — | |
| Gavrilyuk et al.Input=RGB + Optical Flow2021.05 | 55.1 | 50 | 37.6 | 23.1 | 9.4 | 0.4 | 21.5 | 42.6 | — | — | — | |
| Gavrilyuk et al.Modality=RGB+Flow2021.05 | 55.1 | 50 | 37.6 | 23.1 | 9.4 | 0.4 | 21.5 | 42.6 | — | — | — | |
| Gavrilyuk et al.18additional optical flow input=true2022.03 | 55.1 | 50 | 37.6 | 23.1 | 9.4 | 0.4 | 21.5 | 42.6 | — | — | — | |
| Gavrilyuk et al.Pub.=CVPR18, Additional optical flow inputs=true2022.06 | 55.1 | 50 | 37.6 | 23.1 | 9.4 | 0.4 | 21.5 | 42.6 | — | — | — | |
| Gavrilyuk et al.Venue=CVPR2018, additional optical flow input=true2022.04 | 55.1 | 50 | 37.6 | 23.1 | 9.4 | 0.4 | 21.5 | 42.6 | — | — | — | |
| fully-convolutional modelInput modality=RGB2018.03 | 53.6 | 47.5 | 34.7 | 21.1 | 8 | 0.2 | 19.8 | 42.1 | — | — | — | |
| Gavrilyuk et al.Input=RGB2021.05 | 53.6 | 47.5 | 34.7 | 21.1 | 8 | 0.2 | 19.8 | 42.1 | — | — | — | |
| Gavrilyuk et al.Modality=RGB2021.05 | 53.6 | 47.5 | 34.7 | 21.1 | 8 | 0.2 | 19.8 | 42.1 | — | — | — | |
| Gavrilyuk et al.Backbone=I3D2022.01 | 53.6 | 47.5 | 34.7 | 21.1 | 8 | 20 | — | 42.1 | 19.8 | — | — | |
| Gavrilyuk et al.18additional optical flow input=false2022.03 | 53.6 | 47.5 | 34.7 | 21.1 | 8 | 0.2 | 19.8 | 42.1 | — | — | — | |
| Gavrilyuk et al.Input Modality=RGB2021.11 | 53.6 | 47.5 | 34.7 | 21.1 | 8 | 0.2 | — | 42.1 | — | 19.8 | — |