# Measuring User's Mental Models of Speech Translation in Human-AI Collaboration

Source: [Measuring User's Mental Models of Speech Translation in Human-AI Collaboration](<https://www.sota2.com/research/paper/6a3b64de206542251d7c99b1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-25T05:00:34.132Z

Authors: HyoJung Han, Nishant Balepur, Jordan Boyd-Graber, Marine Carpuat

Published: 2026-06-23T14:40:36.000Z

## Abstract

Millions of people use machine translation (MT) tools daily, yet little is known about their perception of what systems can and cannot do. This paper studies users' mental models of speech translation systems through a new framework based on cross-lingual question answering, where users either accept MT output or request professional re-translation to answer questions based on the information presented in a foreign language. By analyzing user behavior and accuracy trends across varying translation qualities, we examine to what extent they can predict where the system is likely to be wrong, and how this mental model evolves. Users develop stronger mental models with practice, especially when they have some knowledge of the source language, primarily by relying on surface-level error cues. Moreover, providing speech transcriptions can help users develop better mental models. Our results show the promise of cross-lingual question answering as a downstream task for studying MT mental models and advancing our understanding of human-AI collaboration.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.24644>)

[PDF](<https://arxiv.org/pdf/2606.24644.pdf>)