Spoken Medical Question Answering on MedQA (test)
97.5QA AccuracyMedSpeak
Evaluation results
| Method | Links | |
|---|---|---|
| MedSpeakTranscription Source=Whisper generated transcripts, Fine-tuning Status=Fine-tuned, Knowledge Graph Context Integration=Budgeted KG context2026.02 | 97.5 | |
| Fine-Tuned LLM + GT (FT-LLM)Transcription Source=Ground truth texts, Fine-tuning Status=Fully fine-tuned, Knowledge Graph Context Integration=No KG-context2026.02 | 91.8 | |
| Fine-Tuned LLM + Whisper (FT+Whisp)Transcription Source=Whisper generated transcripts, Fine-tuning Status=Fine-tuned, Knowledge Graph Context Integration=No KG-context2026.02 | 86.6 | |
| Zero-shot GT (Zero-Shot)Transcription Source=Ground truth transcripts, Fine-tuning Status=No fine-tuning, Knowledge Graph Context Integration=No KG-context2026.02 | 58.9 | |
| Zero-shot ASR (ZS-ASR)Transcription Source=Whisper generated transcript, Fine-tuning Status=No fine-tuning, Knowledge Graph Context Integration=No KG-context2026.02 | 54.9 |