# Exploiting Scene Graphs for Human-Object Interaction Detection Source: [Exploiting Scene Graphs for Human-Object Interaction Detection]() Collection index: [llms.txt]() Data updated: 2025-10-25T23:27:27.413Z Authors: Tao He, Lianli Gao, Jingkuan Song, Yuan-Fang Li Published: 2021-08-19T09:40:50.000Z ## Abstract Human-Object Interaction (HOI) detection is a fundamental visual task aiming at localizing and recognizing interactions between humans and objects. Existing works focus on the visual and linguistic features of humans and objects. However, they do not capitalise on the high-level and semantic relationships present in the image, which provides crucial contextual and detailed relational knowledge for HOI inference. We propose a novel method to exploit this information, through the scene graph, for the Human-Object Interaction (SG2HOI) detection task. Our method, SG2HOI, incorporates the SG information in two ways: (1) we embed a scene graph into a global context clue, serving as the scene-specific environmental context; and (2) we build a relation-aware message-passing module to gather relationships from objects' neighborhood and transfer them into interactions. Empirical evaluation shows that our SG2HOI method outperforms the state-of-the-art methods on two benchmark HOI datasets: V-COCO and HICO-DET. Code will be available at https://github.com/ht014/SG2HOI. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Human-Object Interaction Detection on HICO-DET (test)](): Human-Object Interaction Detection; HICO-DET (test); mAP (full): not recorded. ## Original sources and implementation [Original paper]() [PDF]()