# Visual-Semantic Graph Attention Networks for Human-Object Interaction Detection

Source: [Visual-Semantic Graph Attention Networks for Human-Object Interaction Detection](<https://www.sota2.com/research/paper/68fa696e1e209131cea21fff>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:06:39.448Z

Authors: Zhijun Liang, Juan Rojas, Junfa Liu, Yisheng Guan

Published: 2020-01-07T22:22:46.000Z

## Abstract

In scene understanding, robotics benefit from not only detecting individual scene instances but also from learning their possible interactions. Human-Object Interaction (HOI) Detection infers the action predicate on a \<human, predicate, object\> triplet. Contextual information has been found critical in inferring interactions. However, most works only use local features from single human-object pair for inference. Few works have studied the disambiguating contribution of subsidiary relations made available via graph networks. Similarly, few have learned to effectively leverage visual cues along with the intrinsic semantic regularities contained in HOIs. We contribute a dual-graph attention network that effectively aggregates contextual visual, spatial, and semantic information dynamically from primary human-object relations as well as subsidiary relations through attention mechanisms for strong disambiguating power. We achieve comparable results on two benchmarks: V-COCO and HICO-DET. Code is available at \\url{https://github.com/birlrobotics/vs-gats}.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Human-Object Interaction Detection on HICO-DET (test)](<https://www.sota2.com/research/sota/human-object-interaction-detection-on-hico-det-test.md>): Human-Object Interaction Detection; HICO-DET (test); mAP (full): 20.27.

- [Human-Object Interaction Detection on V-COCO (test)](<https://www.sota2.com/research/sota/human-object-interaction-detection-on-v-coco-test.md>): Human-Object Interaction Detection; V-COCO (test); AP (Role, Scenario 1): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2001.02302>)

[PDF](<https://arxiv.org/pdf/2001.02302.pdf>)