Spoken-Dialect Question Answering on SDQA (test)
49.04Accuracy (USA)AuRA
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AuRALat. (s)=0.40, Mem. (GB)=10.62026.06 | 49.04 | 48.97 | 48.55 | 48.79 | 48.48 | 48.47 | 48.56 | 48.66 | 48.69 | 48.83 | 49.21 | 48.75 | |
| DiVALat. (s)=0.63, Mem. (GB)=18.92026.06 | 47.98 | 47.54 | 44.79 | 47.28 | 44.16 | 47.11 | 48.23 | 47.96 | 45.62 | 45.24 | 43.81 | 46.34 | |
| CascadeLat. (s)=0.94, Mem. (GB)=19.22026.06 | 45.85 | 45.6 | 42.48 | 44.52 | 41.96 | 44.62 | 46.9 | 45.98 | 42.72 | 44.83 | 22.65 | 42.55 | |
| Qwen2.5-OmniLat. (s)=0.52, Mem. (GB)=13.92026.06 | 42.63 | 43.16 | 43.52 | 43.42 | 43.47 | 43.51 | 43.71 | 43.69 | 42.98 | 43.01 | 43.69 | 43.34 | |
| BLSPLat. (s)=0.42, Mem. (GB)=26.52026.06 | 38.46 | 39.07 | 35.84 | 36.39 | 36.59 | 38.64 | 39.95 | 37.6 | 35.44 | 36.62 | 35.13 | 37.25 | |
| Qwen2-AudioLat. (s)=0.57, Mem. (GB)=27.62026.06 | 37.32 | 37.67 | 35.26 | 35.19 | 33.74 | 36.16 | 37.27 | 37.95 | 33.57 | 35.02 | 34.98 | 35.83 |