Spoken Language Understanding on MAC-SLU
4.69Overall AccuracySFL-MTSC
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SFL-MTSCBackbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline), Baseline Prompting=CroPrompt (Slot→Intent)2026.06 | 4.69 | 49.61 | 39.54 | |
| CroPrompt (Slot→Intent)Backbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline)2026.06 | 4.49 | 50.86 | 39.01 | |
| SFL-MTSCBackbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline), Baseline Prompting=GPT-SLU2026.06 | 4.26 | 50.13 | 40.02 | |
| GPT-SLUBackbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline)2026.06 | 4.23 | 53.52 | 43.16 | |
| SFL-MTSCBackbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline), Baseline Prompting=Vanilla Prompting2026.06 | 3.39 | 38.31 | 49.53 | |
| Vanilla PromptingBackbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline)2026.06 | 2.43 | 43.63 | 22.15 | |
| SFL-MTSCBackbone=Qwen2.5-Omni-7B, Inference Type=End-to-End, Baseline Prompting=CroPrompt (Slot→Intent)2026.06 | 2.43 | 6.78 | 3.93 | |
| SFL-MTSCBackbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline), Baseline Prompting=CroPrompt (Intent→Slot)2026.06 | 2.17 | 41.44 | 34.48 | |
| SFL-MTSCBackbone=Qwen2.5-Omni-7B, Inference Type=End-to-End, Baseline Prompting=CroPrompt (Intent→Slot)2026.06 | 2.17 | 18.59 | 4.79 | |
| SFL-MTSCBackbone=Qwen2.5-Omni-7B, Inference Type=End-to-End, Baseline Prompting=GPT-SLU2026.06 | 2.13 | 8.16 | 3.91 | |
| SFL-MTSCBackbone=Qwen2.5-Omni-7B, Inference Type=End-to-End, Baseline Prompting=Vanilla Prompting2026.06 | 2.09 | 13.9 | 4.57 | |
| CroPrompt (Intent→Slot)Backbone=Whisper + Qwen3-4B-Instruct-2507, Inference Type=ASR + NLU (Pipeline)2026.06 | 1.81 | 43.68 | 23.77 | |
| CroPrompt (Slot→Intent)Backbone=Qwen2.5-Omni-7B, Inference Type=End-to-End2026.06 | 0.94 | 12.14 | 0.37 | |
| Vanilla PromptingBackbone=Qwen2.5-Omni-7B, Inference Type=End-to-End2026.06 | 0.64 | 21.46 | 4.86 | |
| CroPrompt (Intent→Slot)Backbone=Qwen2.5-Omni-7B, Inference Type=End-to-End2026.06 | 0.42 | 25.3 | 5.46 | |
| GPT-SLUBackbone=Qwen2.5-Omni-7B, Inference Type=End-to-End2026.06 | 0.18 | 14.12 | 11.24 |