Object Detection on COCO open-vocabulary (test)
39.3Novel APRegionCLIP
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RegionCLIPpretrained model=R-50x4, detector backbone=R-50x4, category=w/ pseudo box labels, notes=uses an off-the-shelf RPN during its pretraining2023.05 | 39.3 | 55.7 | — | — | — | — | |
| RegionCLIPpretrained model=R-50x4, detector backbone=R-50x4, supervision=pseudo box labels, uses off-the-shelf RPN=true2023.09 | 39.3 | 55.7 | — | — | — | — | |
| Rasheed et al.pretrained model=R-50, detector backbone=R-50, category=w/ pseudo box labels, notes=uses an external MViT detector during pretraining2023.05 | 36.9 | 51.5 | — | — | — | — | |
| Rasheed et al.pretrained model=ViT-B/32, detector backbone=R-50, supervision=pseudo box labels, external detector=MViT [41]2023.09 | 36.9 | 51.5 | — | — | — | — | |
| VL-PLMpretrained model=ViT-B/32, detector backbone=R-50, category=w/ pseudo box labels2023.05 | 34.4 | 53.5 | — | — | — | — | |
| VL-PLMpretrained model=ViT-B/32, detector backbone=R-50, supervision=pseudo box labels2023.09 | 34.4 | 53.5 | — | — | — | — | |
| CFM-ViTpretrained model=ViT-L/16, detector backbone=ViT-L/162023.09 | 34.1 | 46 | — | — | — | — | |
| RO-ViTpretrained model=ViT-L/16, detector backbone=ViT-L/16, category=ViT based2023.05 | 33 | 47.7 | — | — | — | — | |
| CFM-ViTpretrained model=ViT-B/16, detector backbone=ViT-B/162023.09 | 30.8 | 42.4 | — | — | — | — | |
| RO-ViTpretrained model=ViT-B/16, detector backbone=ViT-B/16, category=ViT based2023.05 | 30.2 | 41.5 | — | — | — | — | |
| OV-DETRpretrained model=ViT-B/32, detector backbone=R-50, category=ConvNet based2023.05 | 29.4 | 52.7 | — | — | — | — | |
| OV-DETRpretrained model=ViT-B/32, detector backbone=R-502023.09 | 29.4 | 52.7 | — | — | — | — | |
| VILDpretrained model=ViT-B/32, detector backbone=R-50, category=ConvNet based2023.05 | 27.6 | 51.3 | — | — | — | — | |
| ViLDpretrained model=ViT-B/32, detector backbone=R-502023.09 | 27.6 | 51.3 | — | — | — | — | |
| XPM et al.pretrained model=R-50, detector backbone=R-50, category=w/ pseudo box labels2023.05 | 27 | 41.2 | — | — | — | — | |
| XPM et al.pretrained model=R-50, detector backbone=R-50, supervision=pseudo box labels2023.09 | 27 | 41.2 | — | — | — | — | |
| PromptDetpretrained model=ViT-B/32, detector backbone=R-50, category=w/ pseudo box labels2023.05 | 26.6 | 50.6 | — | — | — | — | |
| PromptDetpretrained model=ViT-B/32, detector backbone=R-50, supervision=pseudo box labels2023.09 | 26.6 | 50.6 | — | — | — | — | |
| Detic-CN2pretrained model=ViT-B/32, detector backbone=R-50, category=w/ weak supervision2023.05 | 24.6 | 32.4 | — | — | — | — | |
| Detic-CN2pretrained model=ViT-B/32, detector backbone=R-50, supervision=weak supervision2023.09 | 24.6 | 32.4 | — | — | — | — | |
| LocOvImg-Cap Data Size=0.6M, Evaluation Setup=Constrained2022.05 | 17.2 | — | 30.1 | 33.5 | 53.4 | — | |
| LocOvImg-Cap Data Size=0.6M, Evaluation Setup=Generalized2022.05 | 16.6 | 28.1 | 28.6 | 31.9 | 51.3 | 45.7 | |
| OVRImg-Cap Data Size=0.6M, Evaluation Setup=Constrained2022.05 | 14.6 | — | 27.5 | 26.9 | 46.8 | — | |
| STT-ZSDImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | 0.21 | — | 0.31 | 33.2 | 53.4 | — | |
| STT-ZSDImg-Cap Data Size=-, Evaluation Setup=Generalized2022.05 | 0.03 | 24.4 | 0.05 | 33 | 53.1 | 39.2 | |
| CLIP (cropped reg)Img-Cap Data Size=400M, Evaluation Setup=Generalized2022.05 | — | — | 26.3 | — | 28.3 | 27.8 | |
| DELOImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | — | — | 7.6 | — | 14 | — | |
| DELOImg-Cap Data Size=-, Evaluation Setup=Generalized2022.05 | — | — | 3.41 | — | 13.8 | 13 | |
| DSESImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | — | — | 0.54 | — | 27.2 | — | |
| DSESImg-Cap Data Size=-, Evaluation Setup=Generalized2022.05 | — | — | 0.27 | — | 26.7 | 22.1 | |
| Faster R-CNNImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | — | — | — | — | 54.5 | — | |
| LABImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | — | — | 0.27 | — | 21.1 | — | |
| LABImg-Cap Data Size=-, Evaluation Setup=Generalized2022.05 | — | — | 0.22 | — | 20.8 | 18 | |
| OVRImg-Cap Data Size=0.6M, Evaluation Setup=Generalized2022.05 | — | 22.8 | 22.8 | — | 46 | 39.9 | |
| PLImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | — | — | 10 | — | 36.8 | — | |
| PLImg-Cap Data Size=-, Evaluation Setup=Generalized2022.05 | — | — | 4.12 | — | 35.9 | 27.9 | |
| RegionCLIPImg-Cap Data Size=400.6M, Evaluation Setup=Constrained2022.05 | — | — | 30.8 | — | 55.2 | — | |
| RegionCLIPImg-Cap Data Size=400.6M, Evaluation Setup=Generalized2022.05 | — | — | 26.8 | — | 54.8 | 47.5 | |
| SBImg-Cap Data Size=-, Evaluation Setup=Constrained2022.05 | — | — | 0.7 | — | 29.7 | — | |
| SBImg-Cap Data Size=-, Evaluation Setup=Generalized2022.05 | — | — | 0.31 | — | 29.2 | 24.9 | |
| ViLDImg-Cap Data Size=400M, Evaluation Setup=Generalized2022.05 | — | — | 27.6 | — | 59.5 | 51.3 | |
| XP-MaskImg-Cap Data Size=5.7M, Evaluation Setup=Constrained2022.05 | — | — | 29.9 | — | 46.8 | — | |
| XP-MaskImg-Cap Data Size=5.7M, Evaluation Setup=Generalized2022.05 | — | — | 27 | — | 46.3 | 41.2 |