Relation Detection on VRD (test)
63.4R@50GPS-Net
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPS-Netshared object detector=true, backbone pre-training=COCO2020.03 | 63.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPS-Netshared object detector=true, backbone pre-training=ImageNet2020.03 | 58.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MF-URLN2020.03 | 58.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAI + SCA-Mshared object detector=true2020.03 | 56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KL distillation2020.03 | 55.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S+A] + LanguageSupervision=Full sup., Method components=[S+A] + Language2017.07 | 52.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zoom-Netshared object detector=true2020.03 | 50.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S+A]Supervision=Full sup., Method components=[S+A]2017.07 | 50.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (full)Supervision=Full sup.2017.07 | 48.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Visual + LanguageSupervision=Full sup.2017.07 | 47.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S+A]Supervision=Weak sup., Method components=[S+A]2017.07 | 46.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S+A] - NoisySupervision=Weak sup., Method components=[S+A] - Noisy2017.07 | 46.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [A]Supervision=Full sup., Method components=Appearance [A]2017.07 | 46.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VTransE2020.03 | 44.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S]Supervision=Full sup., Method components=Spatial [S]2017.07 | 42.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPS-Netshared object detector=true, backbone pre-training=COCO2020.03 | 27.8 | 31.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RelDNshared object detector=true2020.03 | 25.3 | 28.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MF-URLN2020.03 | 23.9 | 26.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Yu et. alExternal data=true2019.01 | 22.68 | 31.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPS-Netshared object detector=true, backbone pre-training=ImageNet2020.03 | 21.5 | 24.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VidVRD-IIVisual Feature=RoI, Motion Feature=true2021.12 | 19.63 | 22.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.37 | |
| CAI + SCA-Mshared object detector=true2020.03 | 19.5 | 22.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VTransE2020.03 | 19.4 | 22.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KL distillation2020.03 | 19.2 | 21.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BLOCKExternal data=false2019.01 | 19.06 | 20.96 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zoom-Netshared object detector=true2020.03 | 18.9 | 21.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VRL2020.03 | 18.2 | 20.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (likelihood)Supervision=Full sup.2017.07 | 18.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VRLZero-shot=false2017.03 | 18.19 | 20.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Liang et. alExternal data=false2019.01 | 18.19 | 20.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Dai et. alExternal data=false2019.01 | 17.73 | 20.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Li et. alExternal data=false2019.01 | 17.32 | 20.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ViP-CNN2020.03 | 17.3 | 20 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S+A] + LanguageSupervision=Full sup., Method components=[S+A] + Language2017.07 | 15.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Peyre et. alExternal data=false2019.01 | 15.8 | 17.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S+A]Supervision=Full sup., Method components=[S+A]2017.07 | 14.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BIG-CVisual Feature=RoI (using tracklets/features from MEGA), Motion Feature=true2021.12 | 14.1 | 16.25 | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.08 | |
| Ours [S+A]Supervision=Weak sup., Method components=[S+A]2017.07 | 14.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VTransEModel type=end-to-end2017.02 | 14.07 | 15.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Zhang et. alExternal data=false2019.01 | 14.07 | 15.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Visual + LanguageSupervision=Full sup.2017.07 | 13.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (full)Supervision=Full sup.2017.07 | 13.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lu’s-VLKObject detector=R-CNN, Language priors=word2vec2017.02 | 13.86 | 14.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lu et al.Zero-shot=false2017.03 | 13.86 | 14.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lu et. alExternal data=false2019.01 | 13.86 | 14.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Chen et al.Visual Feature=RoI+I3D, Motion Feature=true2021.12 | 13.73 | 16.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | 20.08 | |
| Ours [S+A] - NoisySupervision=Weak sup., Method components=[S+A] - Noisy2017.07 | 13.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VTransE-2stageModel type=two-stage, Object detector=Faster-RCNN2017.02 | 13.3 | 14.64 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [A]Supervision=Full sup., Method components=Appearance [A]2017.07 | 12.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IVRDVisual Feature=RoI, Motion Feature=true2021.12 | 12.4 | 14.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.97 | |
| Ours [S]Supervision=Full sup., Method components=Spatial [S]2017.07 | 12.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Liu et al.Visual Feature=RoI+I3D (using tracklets/features from [24]), Motion Feature=true2021.12 | 11.21 | 13.69 | — | — | — | — | — | — | — | — | — | — | — | — | — | 18.38 | |
| BIG-CVisual Feature=RoI+I3D (using tracklets/features from [24]), Motion Feature=false2021.12 | 9.63 | 11.29 | — | — | — | — | — | — | — | — | — | — | — | — | — | 17.67 | |
| BIG-CVisual Feature=RoI (using tracklets/features from [24]), Motion Feature=false2021.12 | 9.59 | 10.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | 17.56 | |
| MHAVisual Feature=iDT, Motion Feature=true2021.12 | 9.53 | 10.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | 19.03 | |
| Liu et al.Visual Feature=RoI+I3D (using tracklets/features from [24]), Motion Feature=false2021.12 | 9.14 | 11.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.81 | |
| TRACEVisual Feature=RoI+I3D, Motion Feature=false2021.12 | 9.08 | 11.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | 17.57 | |
| Liu et al.Visual Feature=RoI (using tracklets/features from [24]), Motion Feature=false2021.12 | 8.47 | 11 | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.01 | |
| VRD-GCNVisual Feature=iDT, Motion Feature=true2021.12 | 8.07 | 9.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | 16.26 | |
| VRLZero-shot=true2017.03 | 7.94 | 8.52 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TRACEVisual Feature=RoI, Motion Feature=false2021.12 | 7.67 | 10.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | 15.06 | |
| Random2017.02 | 7.14 | 1.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VisualSupervision=Full sup.2017.07 | 7.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GSTEGVisual Feature=iDT, Motion Feature=true2021.12 | 7.05 | 8.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.52 | |
| VidVRDVisual Feature=iDT, Motion Feature=false2021.12 | 5.54 | 6.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | 8.58 | |
| Faster R-CNN V only2017.03 | 4.26 | 5.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lu et al.Zero-shot=true2017.03 | 3.13 | 3.28 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Joint CNN+Trained RPN2017.03 | 1.98 | 2.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VisualSupervision=Full sup.2017.07 | 1.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lu’s-VObject detector=R-CNN2017.02 | 1.58 | 1.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Lu et al. V only2017.03 | 1.58 | 1.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Joint CNN+RPN2017.03 | 1.15 | 1.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Visual PhrasesSupervision=Full sup.2017.07 | 0.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (likelihood)Supervision=Full sup.2017.07 | 0.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Joint CNN+R-CNN2017.03 | 0.07 | 0.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CAI + SCA-MObject Detector=same as [16]2020.03 | — | — | — | — | — | — | — | — | — | — | 56 | 19.5 | 22.4 | 25.2 | 28.9 | — | |
| CAI+SCA-MPretrained=ImageNet2020.03 | — | — | — | — | 56 | 56 | — | — | — | — | — | — | — | — | — | — | |
| CAI+SCA-MPretrained=ImageNet2020.03 | — | — | — | — | 22.4 | 19.5 | — | — | 28.5 | 22.3 | — | — | — | — | — | — | |
| GPS-NetBackbone=VGG-16 (ImageNet), Object Detector=same as [16]2020.03 | — | — | — | — | — | — | — | — | — | — | 58.7 | 21.5 | 24.3 | 28.9 | 34 | — | |
| GPS-NetBackbone=VGG-16 (COCO), Object Detector=same as [16]2020.03 | — | — | — | — | — | — | — | — | — | — | 63.4 | 27.8 | 31.7 | 33.8 | 39.2 | — | |
| GPS-NetPretrained=ImageNet2020.03 | — | — | — | — | 58.7 | 58.7 | — | — | — | — | — | — | — | — | — | — | |
| GPS-NetPretrained=COCO2020.03 | — | — | — | — | 63.4 | 63.4 | — | — | — | — | — | — | — | — | — | — | |
| GPS-NetPretrained=ImageNet2020.03 | — | — | — | — | 24.3 | 21.5 | — | — | 28.9 | 23.6 | — | — | — | — | — | — | |
| GPS-NetPretrained=COCO2020.03 | — | — | — | — | 31.7 | 27.8 | — | — | 37 | 30.6 | — | — | — | — | — | — | |
| Joint CNNTraining images=VRD only2017.07 | — | — | — | — | 0.09 | 0.07 | — | — | — | — | — | — | — | — | — | — | |
| KL distillation2020.03 | — | — | — | — | — | — | — | — | — | — | 55.2 | 19.2 | 21.3 | 23.1 | 24 | — | |
| KL distillationPretrained=Unknown2020.03 | — | — | — | — | 55.2 | 55.2 | — | — | — | — | — | — | — | — | — | — | |
| KL distillationPretrained=Unknown2020.03 | — | — | — | — | 21.3 | 19.2 | — | — | 31.9 | 22.7 | — | — | — | — | — | — | |
| Language (full)Supervision=Full sup.2017.07 | — | — | 48.4 | 12.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (full)Supervision=Full sup.2017.07 | — | — | 15.3 | 5.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (likelihood)Supervision=Full sup.2017.07 | — | — | 18.2 | 5.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Language (likelihood)Supervision=Full sup.2017.07 | — | — | 0.08 | 0.01 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Linguistic CuesTraining images=VRD only2017.07 | — | — | — | — | — | — | 18.37 | 15.08 | — | — | — | — | — | — | — | — | |
| MF-URLN2020.03 | — | — | — | — | — | — | — | — | — | — | 58.2 | 23.9 | 26.8 | 31.5 | 36.1 | — | |
| MF-URLNPretrained=Unknown2020.03 | — | — | — | — | 58.2 | 58.2 | — | — | — | — | — | — | — | — | — | — | |
| MF-URLNPretrained=Unknown2020.03 | — | — | — | — | 26.8 | 23.9 | — | — | — | — | — | — | — | — | — | — | |
| Ours [A]Supervision=Full sup., Features=Appearance2017.07 | — | — | 46.3 | 16.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [A]Supervision=Full sup., Features=Appearance2017.07 | — | — | 14.4 | 5.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S]Supervision=Full sup., Features=Spatial2017.07 | — | — | 42.2 | 22.2 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ours [S]Supervision=Full sup., Features=Spatial2017.07 | — | — | 13.5 | 8.1 | — | — | — | — | — | — | — | — | — | — | — | — |