# Benchmarking Humans and Machines on Complex Multilingual Speech Understanding Tasks

Source: [Benchmarking Humans and Machines on Complex Multilingual Speech Understanding Tasks](<https://www.sota2.com/research/paper/68fa807e1e209131cea9d42d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-12T05:00:04.726Z

Authors: Sai Samrat Kankanala, Ram Chandra, Sriram Ganapathy

Published: 2025-09-22T16:18:05.000Z

## Abstract

Auditory attention and selective phase-locking are central to human speech understanding in complex acoustic scenes and cocktail party settings, yet these capabilities in multilingual subjects remain poorly understood. While machine understanding of natural speech has advanced in recent years, questions persist about comprehension of overlapped and mixed-channel speech. We propose a systematic paradigm for studying humans and machines in speech question-answering tasks in multilingual settings with clean and mixed-channel speech. For human listeners, selective attention to a target speaker was significantly better in their native language (L1) than in their second language (L2). For machine listening, speech-based large language models (LLMs) match or exceed human performance in clean, single-speaker conditions but often struggle to selectively attend in two-speaker settings. These results reveal a key divergence: humans rely on attentional cues that are more streamlined in their native language, whereas LLMs default to parallel information extraction which exceed human skills.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2509.17965>)

[PDF](<https://arxiv.org/pdf/2509.17965.pdf>)
