# From Audio to Semantics: Approaches to end-to-end spoken language understanding

Source: [From Audio to Semantics: Approaches to end-to-end spoken language understanding](<https://www.sota2.com/research/paper/68fa69501e209131cea21792>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:56:46.471Z

Authors: Parisa Haghani, Arun Narayanan, Michiel Bacchiani, Galen Chuang, Neeraj Gaur, Pedro Moreno, Rohit Prabhavalkar, Zhongdi Qu, Austin Waters

Published: 2018-09-24T19:46:24.000Z

## Abstract

Conventional spoken language understanding systems consist of two main components: an automatic speech recognition module that converts audio to a transcript, and a natural language understanding module that transforms the resulting text (or top N hypotheses) into a set of domains, intents, and arguments. These modules are typically optimized independently. In this paper, we formulate audio to semantic understanding as a sequence-to-sequence problem \[1\]. We propose and compare various encoder-decoder based approaches that optimize both modules jointly, in an end-to-end manner. Evaluations on a real-world task show that 1) having an intermediate text representation is crucial for the quality of the predicted semantics, especially the intent arguments and 2) jointly optimizing the full system improves overall accuracy of prediction. Compared to independently trained models, our best jointly trained model achieves similar domain and intent prediction F1 scores, but improves argument word error rate by 18% relative.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Spoken Language Understanding on Internal Traffic Dataset (regular test)](<https://www.sota2.com/research/sota/spoken-language-understanding-on-internal-traffic-dataset-regular-test.md>): Spoken Language Understanding; Internal Traffic Dataset (regular test); ICER Improvement (%): 4.

- [Spoken Language Understanding on Internal Traffic Dataset hard (test)](<https://www.sota2.com/research/sota/spoken-language-understanding-on-internal-traffic-dataset-hard-test.md>): Spoken Language Understanding; Internal Traffic Dataset hard (test); ICER (Relative Improvement): 4.4.

- [Spoken Language Understanding on in-house SLU dataset](<https://www.sota2.com/research/sota/spoken-language-understanding-on-in-house-slu-dataset.md>): Spoken Language Understanding; in-house SLU dataset; RI-ICER: 30.

- [Spoken Language Understanding on FSC (test)](<https://www.sota2.com/research/sota/spoken-language-understanding-on-fsc-test.md>): Spoken Language Understanding; FSC (test); Intent Accuracy: 99.13.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1809.09190>)

[PDF](<https://arxiv.org/pdf/1809.09190.pdf>)