Prosody Sensitivity Evaluation on ContraProST
14PA (German)SpeechLLM +FT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SpeechLLM +FTInput Modality=TEXT, Evaluation Protocol=fine-tuned2026.05 | 14 | 14 | 26.1 | |
| SpeechLLM +FTInput Modality=SP.+TXT, Evaluation Protocol=fine-tuned2026.05 | 17.8 | 8.8 | 20.4 | |
| SpeechLLM +FTInput Modality=SPEECH, Evaluation Protocol=fine-tuned2026.05 | 21.1 | 21.9 | 15.7 | |
| SpeechLLMInput Modality=TEXT, Evaluation Protocol=zero-shot2026.05 | 23.8 | 18.1 | 31.1 | |
| SPEECHQEInput Modality=Speech2026.05 | 26.1 | 20 | 38.2 | |
| SpeechLLMInput Modality=SP.+TXT, Evaluation Protocol=zero-shot2026.05 | 31 | 26.1 | 32.6 | |
| SpeechLLMInput Modality=SPEECH, Evaluation Protocol=zero-shot2026.05 | 32.2 | 28.4 | 33.1 | |
| COMETKIWIInput Modality=TEXT2026.05 | 50 | 50 | 50 | |
| COMETKiwiWMT RoBERTaInput Modality=TEXT2026.05 | 50 | 50 | 50 | |
| COMETKiwiIWSLT RoBERTaInput Modality=TEXT2026.05 | 50 | 50 | 50 | |
| SpeechCOMETSONARInput Modality=SPEECH2026.05 | 50 | 50 | 50 | |
| SpeechCOMETWhisperInput Modality=SPEECH2026.05 | 50 | 50 | 50.1 | |
| SpeechCOMETInput Modality=SP.+TXT2026.05 | 50 | 49.9 | 50 | |
| SpeechCOMETXLInput Modality=SP.+TXT2026.05 | 50 | 50 | 50 | |
| COMET-PARTIALInput Modality=Text2026.05 | 50 | 50 | 50 | |
| COMETKIWIInput Modality=Text2026.05 | 50 | 50 | 50 | |
| BLASERInput Modality=Speech2026.05 | 51 | 51.5 | 51.1 |