# Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization Source: [Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization]() Collection index: [llms.txt]() Data updated: 2026-06-09T05:00:07.007Z Authors: Zhe Yang, Ruyi Zhang, Hongtao Chen, Wenrui Li, Hengyu Man, Wangmeng Zuo, Xiaopeng Fan Published: 2026-06-05T08:23:58.000Z ## Abstract Open-vocabulary audio-visual event localization (OV-AVEL) jointly models audio-visual cues to recognize and temporally localize events, including categories unseen during training. Existing methods primarily learn joint audio-visual representations in Euclidean space, but still face two significant challenges. First, the lack of supervision signals for unseen categories makes it difficult to maintain audio-visual consistency across multiple temporal scales. Second, the lack of hierarchical constraints between segment- and video-level semantics prevents the model from establishing semantic consistency across different levels. To address these challenges, we propose a hierarchical semantic constrained heterogeneous graph (HSCHG) for audio-visual event localization framework. We first construct a heterogeneous hierarchical graph in Euclidean space, which includes audio and visual segment nodes and their corresponding video-level nodes. We use multi-directional temporal edges to capture complete temporal information within each modality. Simultaneously, we employ a dual-threshold filtering gated fusion strategy, introducing cross-modal information only when the alignment confidence is high. Furthermore, we introduce bidirectional semantic constraints between segment- and video-level representations to achieve semantic consistency across different levels. Based on this, we map the multi-level audio-visual representations and text prototypes uniformly into hyperbolic space. We use a hierarchical entailment regularization loss to characterize the hierarchical relationships between videos and segments. Extensive experimental results show that our method outperforms existing methods on the OV-AVEL benchmark. Ablation studies further validate the effectiveness of our method. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Open-Vocabulary Audio-Visual Event Localization on OV-AVEBench (unseen)](): Open-Vocabulary Audio-Visual Event Localization; OV-AVEBench (unseen); Accuracy: 65.6. - [Audio-Visual Event Localization on OV-AVEBench (Seen)](): Audio-Visual Event Localization; OV-AVEBench (Seen); Accuracy: 76.4. ## Original sources and implementation [Original paper]() [PDF]()