Spoken Intelligence Evaluation on LLM_Voice 1.0 (test)
55.9Remembering ScoreCanary 1B + Qwen2 7B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Canary 1B + Qwen2 7BApproach Type=ASR+LLM, ASR Component=Canary 1B, LLM Component=Qwen2 7B2025.07 | 55.9 | 56.6 | 50.4 | 107.78 | |
| Whisperv2-1.5B + Qwen2 7BApproach Type=ASR+LLM, ASR Component=Whisperv2-1.5B, LLM Component=Qwen2 7B2025.07 | 55.4 | 49.9 | 48.1 | 107.43 | |
| Whisperv3-1.5B + Qwen2 7BApproach Type=ASR+LLM, ASR Component=Whisperv3-1.5B, LLM Component=Qwen2 7B2025.07 | 55.3 | 43.3 | 43.2 | 106.49 | |
| Whisperv2-1.5B + GPT-4o + Qwen2 7BApproach Type=ASR+GER+LLM, ASR Component=Whisperv2-1.5B, GER Component=GPT-4o, LLM Component=Qwen2 7B2025.07 | 54.3 | 63.2 | 48.7 | 108.64 | |
| OWSM-CTCV3.1-1B + Qwen2 7BApproach Type=ASR+LLM, ASR Component=OWSM-CTCV3.1-1B, LLM Component=Qwen2 7B2025.07 | 53.4 | 15.1 | 35.3 | 103.05 | |
| Salmonn 13BApproach Type=Multi-Modal End-to-End, Whisper Component=1.5B2025.07 | 50.8 | 38.1 | -114.6 | 101.03 | |
| Gemini-1.5-proApproach Type=Multi-Modal End-to-End2025.07 | 49.2 | 40.9 | 71 | 107.08 | |
| Qwen2.5-Omni 7BApproach Type=Multi-Modal End-to-End, Whisper Component=1.5B2025.07 | 47.2 | 41 | 50.9 | 105.74 | |
| Baichuan-omni-1.5 7BApproach Type=Multi-Modal End-to-End2025.07 | 44.8 | 18.4 | 54.6 | 104.02 | |
| AnyGPT 7BApproach Type=Multi-Modal End-to-End2025.07 | 31.4 | -271.8 | -289.3 | 60.02 | |
| Qwen2-Audio 7BApproach Type=Multi-Modal End-to-End, Whisper Component=1.5B2025.07 | -18.7 | 36.6 | 1.1 | 103.88 | |
| Gemini-1.5-flashApproach Type=Multi-Modal End-to-End2025.07 | -188.5 | 64.1 | 67.3 | 107.85 | |
| Desta 28BApproach Type=Multi-Modal End-to-End, Whisper Component=1.5B2025.07 | -257.5 | -160.4 | -23.3 | 79.69 |