Function Calling on in-house product dataset
95.94Exact Match (EM)CORTIS
Evaluation Results
| Method | Links | |
|---|---|---|
| CORTISScale=3B, Speech input condition=Oracle2026.06 | 95.94 | |
| ASR–LLM CascadeScale=3B, Speech input condition=Oracle2026.06 | 95.43 | |
| CORTISScale=7B, Speech input condition=Oracle2026.06 | 94.42 | |
| ASR–LLM CascadeScale=7B, Speech input condition=Oracle2026.06 | 93.91 | |
| ASR–LLM CascadeScale=3B, Speech input condition=Clean2026.06 | 90.86 | |
| CORTISScale=7B, Speech input condition=Clean2026.06 | 90.19 | |
| CORTISScale=7B, Speech input condition=15 dB2026.06 | 89.68 | |
| ASR–LLM CascadeScale=7B, Speech input condition=Clean2026.06 | 89.68 | |
| CORTISScale=3B, Speech input condition=Clean2026.06 | 89.51 | |
| ASR–LLM CascadeScale=3B, Speech input condition=15 dB2026.06 | 89 | |
| CORTISScale=3B, Speech input condition=15 dB2026.06 | 88.32 | |
| ASR–LLM CascadeScale=7B, Speech input condition=15 dB2026.06 | 86.97 | |
| CORTISScale=7B, Speech input condition=10 dB2026.06 | 86.8 | |
| ASR–LLM CascadeScale=3B, Speech input condition=10 dB2026.06 | 85.28 | |
| CORTISScale=3B, Speech input condition=10 dB2026.06 | 84.77 | |
| ASR–LLM CascadeScale=7B, Speech input condition=10 dB2026.06 | 83.93 | |
| CORTISScale=7B, Speech input condition=5 dB2026.06 | 76.82 | |
| CORTISScale=3B, Speech input condition=5 dB2026.06 | 75.3 | |
| ASR–LLM CascadeScale=7B, Speech input condition=5 dB2026.06 | 73.6 | |
| ASR–LLM CascadeScale=3B, Speech input condition=5 dB2026.06 | 72.59 | |
| CORTISScale=7B, Speech input condition=2.5 dB2026.06 | 67.34 | |
| CORTISScale=3B, Speech input condition=2.5 dB2026.06 | 64.3 | |
| ASR–LLM CascadeScale=7B, Speech input condition=2.5 dB2026.06 | 62.1 | |
| ASR–LLM CascadeScale=3B, Speech input condition=2.5 dB2026.06 | 60.07 | |
| CORTISScale=7B, Speech input condition=0 dB2026.06 | 55.33 | |
| CORTISScale=3B, Speech input condition=0 dB2026.06 | 49.24 | |
| ASR–LLM CascadeScale=7B, Speech input condition=0 dB2026.06 | 44.84 | |
| ASR–LLM CascadeScale=3B, Speech input condition=0 dB2026.06 | 44.67 |