# Approaching Dialogue State Tracking via Aligning Speech Encoders and LLMs Source: [Approaching Dialogue State Tracking via Aligning Speech Encoders and LLMs]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:54:52.186Z Authors: \\v{S}imon Sedl\\'a\\v{c}ek, Bolaji Yusuf, J\\'an \\v{S}vec, Pradyoth Hegde, Santosh Kesiraju, Old\\v{r}ich Plchot, Jan \\v{C}ernock\\'y Published: 2025-06-10T09:46:29.000Z ## Abstract In this work, we approach spoken Dialogue State Tracking (DST) by bridging the representation spaces of speech encoders and LLMs via a small connector module, with a focus on fully open-sourced and open-data components (WavLM-large, OLMo). We focus on ablating different aspects of such systems including full/LoRA adapter fine-tuning, the effect of agent turns in the dialogue history, as well as fuzzy matching-based output post-processing, which greatly improves performance of our systems on named entities in the dialogue slot values. We conduct our experiments on the SpokenWOZ dataset, and additionally utilize the Speech-Aware MultiWOZ dataset to augment our training data. Ultimately, our best-performing WavLM + connector + OLMo-1B aligned models achieve state of the art on the SpokenWOZ test set (34.66% JGA), and our system with Gemma-2-9B-instruct further surpasses this result, reaching 42.17% JGA on SpokenWOZ test. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Dialog State Tracking on SpokenWoz (test)](): Dialog State Tracking; SpokenWoz (test); JGA: 42.17. ## Original sources and implementation [Original paper]() [PDF]()