# ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interaction

Source: [ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interaction](<https://www.sota2.com/research/paper/691566140052cac1dc65d8c4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-10T05:00:05.424Z

Authors: Shu-wen Yang, Ming Tu, Andy T. Liu, Xinghua Qu, Hung-yi Lee, Lu Lu, Yuxuan Wang, Yonghui Wu

Published: 2025-11-11T19:29:51.000Z

## Abstract

Speech-to-Speech (S2S) models have shown promising dialogue capabilities, but their ability to handle paralinguistic cues - such as emotion, tone, and speaker attributes - and to respond appropriately in both content and style remains under-explored. Progress is further hindered by the scarcity of high-quality and expressive demonstrations. To address this, we introduce a new reinforcement learning (RL) framework for paralinguistic-aware S2S, ParaS2S, which evaluates and optimizes both response content and speaking style directly at the waveform level. We first construct ParaS2SBench, a benchmark that evaluates the naturalness of input-output pairs in terms of content and speaking style using expressive and challenging queries. For the automatic judge, we propose a PolyTone training strategy and a multi-stage framework, preventing the style hallucination of end-to-end audio LLM judging. Our judge correlates well with human preferences and is scalable, enabling the model to interact and learn from unlabeled speech via RL. Experiments show that existing S2S models fail to respond appropriately to paralinguistic attributes, performing no better than pipeline-based baselines. Our RL approach (ParaS2SAlign) achieves a 10% relative improvement in the appropriateness of response content and speaking style on ParaS2SBench over supervised fine-tuning (SFT), surpassing all prior models while requiring substantially fewer paired demonstrations than pure SFT. Our findings highlight the need for a scalable and accurate automatic evaluator for speech-to-speech interaction.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Paralinguistic-aware dialogue on ParaS2SBench](<https://www.sota2.com/research/sota/paralinguistic-aware-dialogue-on-paras2sbench.md>): Paralinguistic-aware dialogue; ParaS2SBench; Synthetic Age Score: 4.496.

- [Paralinguistic-aware dialogue on ParaS2SBench Real (Human Evaluation)](<https://www.sota2.com/research/sota/paralinguistic-aware-dialogue-on-paras2sbench-real-human-evaluation.md>): Paralinguistic-aware dialogue; ParaS2SBench Real (Human Evaluation); IEMOCAP Score: 4.336.

- [Paralinguistic-aware dialogue on ParaS2SBench Synthetic (Human Evaluation)](<https://www.sota2.com/research/sota/paralinguistic-aware-dialogue-on-paras2sbench-synthetic-human-evaluation.md>): Paralinguistic-aware dialogue; ParaS2SBench Synthetic (Human Evaluation); Average Paralinguistic Awareness Score: 4.407.

- [Response Ranking on ParaS2SBench](<https://www.sota2.com/research/sota/response-ranking-on-paras2sbench.md>): Response Ranking; ParaS2SBench; Age Score: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.08723>)

[PDF](<https://arxiv.org/pdf/2511.08723.pdf>)