Auditory Reasoning on TAU Benchmark 25 (test)
72.4Single-hop AccGemini 2.5 Pro (Teacher)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 2.5 Pro (Teacher)2026.03 | 72.4 | 73.9 | 73 | |
| Gemini 2.5 Flash2026.03 | 61.3 | 63.2 | 62.1 | |
| Tai-LALMParams=8.8B, Backbone=Llama-3-8B-Instruct, Fine-tuning=LoRA (r=32, a=32)2026.03 | 49.4 | 48.8 | 49.1 | |
| Negative Control (Single-ASR)Params=8.8B, Protocol=Direct fine-tuning on Unfiltered Raw TW-Sound580K+2026.03 | 46.8 | 45.8 | 46.4 | |
| Qwen2.5-Omni-7BParams=7.6B2026.03 | 46.4 | 46.1 | 46.3 | |
| DeSTA 2.5-AudioParams=8.8B2026.03 | 43.3 | 41.7 | 42.6 | |
| Qwen2-Audio + TW-Sound580KParams=8.2B2026.03 | 33 | 31.8 | 32.5 | |
| Qwen2-Audio-InstructParams=8.2B2026.03 | 30.3 | 27.8 | 29.3 | |
| Gemma-3n-E2B-itParams=4.4B2026.03 | 29.6 | 25.8 | 28 |