# Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning

Source: [Weakly-supervised HOI Detection via Prior-guided Bi-level Representation Learning](<https://www.sota2.com/research/paper/68fa6eaa1e209131cea413ce>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:02:50.787Z

Authors: Bo Wan, Yongfei Liu, Desen Zhou, Tinne Tuytelaars, Xuming He

Published: 2023-03-02T14:41:31.000Z

## Abstract

Human object interaction (HOI) detection plays a crucial role in human-centric scene understanding and serves as a fundamental building-block for many vision tasks. One generalizable and scalable strategy for HOI detection is to use weak supervision, learning from image-level annotations only. This is inherently challenging due to ambiguous human-object associations, large search space of detecting HOIs and highly noisy training signal. A promising strategy to address those challenges is to exploit knowledge from large-scale pretrained models (e.g., CLIP), but a direct knowledge distillation strategy~\\citep{liao2022gen} does not perform well on the weakly-supervised setting. In contrast, we develop a CLIP-guided HOI representation capable of incorporating the prior knowledge at both image level and HOI instance level, and adopt a self-taught mechanism to prune incorrect human-object associations. Experimental results on HICO-DET and V-COCO show that our method outperforms the previous works by a sizable margin, showing the efficacy of our HOI representation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Human-Object Interaction Detection on HICO-DET](<https://www.sota2.com/research/sota/human-object-interaction-detection-on-hico-det.md>): Human-Object Interaction Detection; HICO-DET; mAP (Full): 22.89.

- [Human-Object Interaction Detection on V-COCO (test)](<https://www.sota2.com/research/sota/human-object-interaction-detection-on-v-coco-test.md>): Human-Object Interaction Detection; V-COCO (test); AP (Role, Scenario 1): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2303.01313>)

[PDF](<https://arxiv.org/pdf/2303.01313.pdf>)