# MMAudio-LABEL: Audio Event Labeling via Audio Generation for Silent Video

Source: [MMAudio-LABEL: Audio Event Labeling via Audio Generation for Silent Video](<https://www.sota2.com/research/paper/69f827dac0f56ed843088845>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-05T05:00:04.460Z

Authors: Kazuya Tateishi, Akira Takahashi, Atsuo Hiroe, Hirofumi Takeda, Shusuke Takahashi, Yuki Mitsufuji

Published: 2026-05-01T08:09:06.000Z

## Abstract

Recent advances in multimodal generation have enabled high-quality audio generation from silent videos. Practical applications, such as sound production, demand not only the generated audio but also explicit sound event labels detailing the type and timing of sounds. One straightforward approach involves applying a standard sound event detection to the generated audio. However, this post-hoc pipeline is inherently limited, as it is prone to error accumulation. To address this limitation, we propose MMAudio-LABEL (LAtent-Based Event Labeling), an event-aware audio generation framework built on a foundational audio generation model as its backbone that jointly generates audio and frame-aligned sound event predictions from silent videos. We evaluate our method on the Greatest Hits dataset for onset detection and 17-class material classification. Our approach improves onset-detection accuracy from 46.7% to 75.0% and material-classification accuracy from 40.6% to 61.0% over baselines. These results suggest that jointly learning audio generation and event prediction enables a more interpretable and practical video-to-audio synthesis.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Onset detection on Greatest Hits](<https://www.sota2.com/research/sota/onset-detection-on-greatest-hits.md>): Onset detection; Greatest Hits; Count Match: 54.6.

- [Material Classification on Greatest Hits](<https://www.sota2.com/research/sota/material-classification-on-greatest-hits.md>): Material Classification; Greatest Hits; Accuracy: 61.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.00495>)

[PDF](<https://arxiv.org/pdf/2605.00495.pdf>)