Multiple-choice Question Answering on AirBench Foundational
44Total Average ScoreQwen2-Audio Instruct
Evaluation results
| Method | Links | |||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2-Audio InstructArchitecture=Audio Encoder-LLM Decoder, Parameter count=8400M (100x), Number of audio tokens=7502026.01 | 44 | 49.8 | 17.8 | 71.1 | 40.5 | 62.8 | 46.1 | 49.6 | 63.9 | 24.3 | 24.7 | 56 | 38.7 | 43.5 | 26.3 | 38.1 | 52.5 | 35.4 | 22.3 | 48.3 | 78 | 38 | 51.9 | |
| WavLink-BaseArchitecture=Dual Encoder, Parameter count=84M, Number of audio tokens=12026.01 | 42 | 48.3 | 26 | 71.3 | 44.2 | 50.8 | 47.9 | 62.5 | 64 | 26 | 27.8 | 33.1 | 40.7 | 34.4 | 29.2 | 27.1 | 53.3 | 37.3 | 13.5 | 25.3 | 31.4 | 54.8 | 16.2 | |
| Falcon3-Audio 3BArchitecture=Audio Encoder-LLM Decoder, Parameter count=3600M (43x), Number of audio tokens=7502026.01 | 42 | 53.4 | 60 | 74.9 | 40.8 | 52.4 | 42.2 | 46.9 | 59.9 | 19.6 | 22.8 | 41.8 | 39.9 | 35.1 | 20.3 | 4.9 | 23.9 | 60.3 | 26.4 | 26.2 | 51.4 | 39.2 | 49.3 | |
| LAION-CLAPArchitecture=Dual Encoder, Parameter count=193M (2x), Number of audio tokens=12026.01 | 35.8 | 42.6 | 20.8 | 55.9 | 44.8 | 43.5 | 46.2 | 56.2 | 64.5 | 31.4 | 25.6 | 29.5 | 38.6 | 24.7 | 25.4 | 19.3 | 35.9 | 19.7 | 25.3 | 22.5 | 27 | 21.4 | 20.1 |