Zero-Shot Human-Object Interaction Detection on HICO-DET
20.27mAP (Default Unseen)RLIP-ParSe
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RLIP-ParSeZero-shot Protocol=UC-NF2022.09 | 20.27 | 27.67 | — | — | — | — | — | — | — | 26.19 | — | |
| ParSeZero-shot Protocol=UC-NF2022.09 | 19.65 | 24.5 | — | — | — | — | — | — | — | 23.38 | — | |
| RLIP-ParSeZero-shot Protocol=UC-RF2022.09 | 19.19 | 33.35 | — | — | — | — | — | — | — | 30.52 | — | |
| FCLZero-shot Protocol=UC-NF2022.09 | 18.66 | 19.55 | — | — | — | — | — | — | — | 19.37 | — | |
| ParSeZero-shot Protocol=UC-RF2022.09 | 18.53 | 32.21 | — | — | — | — | — | — | — | 29.06 | — | |
| ATLZero-shot Protocol=UC-NF2022.09 | 18.25 | 18.78 | — | — | — | — | — | — | — | 18.67 | — | |
| SCLK (number of selected unknown concepts)=all2022.03 | 17.19 | 19.45 | 19.07 | — | — | — | 100 | — | — | — | — | |
| VCLstrategy=non-rare first, backbone=ResNet-101, finetuned object detector=true2020.07 | 16.22 | 18.52 | 18.06 | 20.93 | 21.02 | 20.9 | — | — | — | — | — | |
| VCLZero-shot Protocol=UC-NF2022.09 | 16.22 | 18.52 | — | — | — | — | — | — | — | 18.06 | — | |
| PD-NetZero-shot=true, Classifier=Clustering-based object-specific (CSP)2020.08 | 15.95 | 18.8 | 18.23 | — | — | — | — | — | — | — | — | |
| PD-NetClassifier=Clustering-based Object-Specific (CSP) verb classifiers2020.08 | 15.49 | 17.07 | 16.8 | — | — | — | — | — | — | — | — | |
| PD-Net (SH)Zero-shot=true, Classifier=object-shared (SH)2020.08 | 15.28 | 17.41 | 16.98 | — | — | — | — | — | — | — | — | |
| PD-Net (SH)Classifier=Object-shared (SH) verb classifiers2020.08 | 14.85 | 16.89 | 16.55 | — | — | — | — | — | — | — | — | |
| SCLK (number of selected unknown concepts)=5002022.03 | 13.91 | 19.45 | 18.53 | — | — | — | 56 | — | — | — | — | |
| SCLK (number of selected unknown concepts)=6002022.03 | 13.91 | 19.45 | 18.53 | — | — | — | 56 | — | — | — | — | |
| SCLK (number of selected unknown concepts)=3002022.03 | 13.52 | 19.45 | 18.46 | — | — | — | 52 | — | — | — | — | |
| SCLK (number of selected unknown concepts)=4002022.03 | 13.52 | 19.45 | 18.46 | — | — | — | 52 | — | — | — | — | |
| FCLZero-shot Protocol=UC-RF2022.09 | 13.16 | 24.23 | — | — | — | — | — | — | — | 22.01 | — | |
| SCLK (number of selected unknown concepts)=2002022.03 | 12.4 | 19.45 | 18.28 | — | — | — | 48 | — | — | — | — | |
| SCLK (number of selected unknown concepts)=1002022.03 | 11.41 | 19.45 | 18.11 | — | — | — | 41 | — | — | — | — | |
| Bansal et al.Zero-shot=true2020.08 | 11.31 | 12.74 | 12.45 | — | — | — | — | — | — | — | — | |
| Bansal et al.Setting=Zero-shot HOI detection2020.08 | 11.22 | 14.36 | 13.84 | — | — | — | — | — | — | — | — | |
| Bansal et al.backbone=ResNet-101, finetuned object detector=true2020.07 | 10.93 | 12.6 | 12.26 | — | — | — | — | — | — | — | — | |
| VCLstrategy=rare first, backbone=ResNet-101, finetuned object detector=true2020.07 | 10.06 | 24.28 | 21.43 | 12.12 | 26.71 | 23.79 | — | — | — | — | — | |
| VCLZero-shot Protocol=UC-RF2022.09 | 10.06 | 24.28 | — | — | — | — | — | — | — | 21.43 | — | |
| ATLZero-shot Protocol=UC-RF2022.09 | 9.18 | 24.67 | — | — | — | — | — | — | — | 21.57 | — | |
| VCLstrategy=non-rare first2020.07 | 9.13 | 13.67 | 12.76 | 12.97 | 16.31 | 15.64 | — | — | — | — | — | |
| VCLstrategy=rare first2020.07 | 7.55 | 18.84 | 16.58 | 10.66 | 21.56 | 19.38 | — | — | — | — | — | |
| Shen et al.2020.07 | 5.62 | — | 6.26 | — | — | — | — | — | — | — | — | |
| Shen et al.Zero-shot=true2020.08 | 5.62 | — | 6.26 | — | — | — | — | — | — | — | — | |
| w/o VCLstrategy=non-rare first2020.07 | 5.06 | 12.77 | 11.23 | 8.81 | 15.37 | 14.06 | — | — | — | — | — | |
| SCLK (number of selected unknown concepts)=02022.03 | 3.92 | 19.45 | 16.86 | — | — | — | 0 | — | — | — | — | |
| w/o VCLstrategy=rare first2020.07 | 3.3 | 18.63 | 15.56 | 5.53 | 21.3 | 18.15 | — | — | — | — | — | |
| ATLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 9.18 | |
| ATLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 24.67 | |
| ATLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 21.57 | |
| CMD-SEPretrained Detector=false, Method Setting=Open-vocabulary Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 16.7 | |
| CMD-SEPretrained Detector=false, Method Setting=Open-vocabulary Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 23.95 | |
| CMD-SEPretrained Detector=false, Method Setting=Open-vocabulary Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 22.35 | |
| FCLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 13.16 | |
| FCLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 24.23 | |
| FCLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 22.01 | |
| GEN-VLKTPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 21.36 | |
| GEN-VLKTPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 32.91 | |
| GEN-VLKTPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 30.56 | |
| HOICLIPPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 23.48 | |
| HOICLIPPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 34.47 | |
| HOICLIPPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 32.26 | |
| MDETR-ParSeZero-shot Protocol=NF2022.09 | — | — | — | — | — | — | — | 0 | 0 | 0 | — | |
| RLIP-ParSeZero-shot Protocol=NF2022.09 | — | — | — | — | — | — | — | 15.08 | 15.5 | 15.4 | — | |
| THIDPretrained Detector=false, Method Setting=Open-vocabulary Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 15.53 | |
| THIDPretrained Detector=false, Method Setting=Open-vocabulary Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 24.32 | |
| THIDPretrained Detector=false, Method Setting=Open-vocabulary Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 22.38 | |
| VCLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 10.06 | |
| VCLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 24.28 | |
| VCLPretrained Detector=true, Method Setting=Zero-shot Methods2024.04 | — | — | — | — | — | — | — | — | — | — | 21.43 |