List your company
ResearchFormerly WizWand

Dialog State Tracking on SpokenWoz (test)

45.52JGAFull Spoken Context + Gemma2-9B-Instruct
17.627224.868632.1139.3514May 19, 2023Oct 19, 2023Mar 21, 2024Aug 22, 2024Jan 23, 2025Jun 26, 2025Nov 27, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Full Spoken Context + Gemma2-9B-InstructSetup=potentially data-contaminated models2025.1045.52
Compressed Spoken Context + Gemma2-9B-InstructSetup=potentially data-contaminated models2025.1043.16
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=Gemma-3-12B-it, Training Text=MW2025.1143
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=Gemma-3-12B-it, Training Text=✗2025.1142.6
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=Gemma-3-12B-it, Training Text=SW2025.1142.2
WavLM + conn. + Gemma2-9B-InstructSetup=potentially data-contaminated models2025.1042.17
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=Gemma-3-4B-it, Training Text=✗2025.1141
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=Gemma-3-4B-it, Training Text=MW2025.1140.3
Full Spoken ContextSetup=open data models2025.1039.32
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=Gemma-3-4B-it, Training Text=SW2025.1137.4
Compressed Spoken ContextSetup=open data models2025.1036.49
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=Gemma-3-1B-it, Training Text=MW2025.1136.3
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=Gemma-3-1B-it, Training Text=✗2025.1135.6
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=OLMo-1B, Training Text=MW2025.1134.9
WavLM + conn. + OLMo-1BSetup=open data models2025.1034.66
Joint Speech and Text E2E DSTTraining Speech=SW, LLM=OLMo-1B, Training Text=✗2025.1134.1
OLMo-1B [7]Training Speech=SW, LLM=OLMo-1B, Training Text=✗2025.1132.1
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=Gemma-3-1B-it, Training Text=SW2025.1130.9
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=OLMo-1B, Training Text=SW2025.1129.3
UBAR + GenWOZSetup=open data models2025.1025.9
SPACE+WavLMalignSetup=open data models2025.1025.65
E2E (Whisper+T5)Setup=open data models2025.1024.1
SPECTRAmode=Fine-tuning2023.0521.96
SPACE+WavLM+TripPymode=Fine-tuning2023.0520.9
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=Gemma-3-12B-it, Training Text=✗2025.1120.7
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=Gemma-3-4B-it, Training Text=✗2025.1119.8
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=Gemma-3-1B-it, Training Text=✗2025.1118.9
Joint Speech and Text E2E DSTTraining Speech=MW, LLM=OLMo-1B, Training Text=✗2025.1118.7