Audio-to-Text Retrieval on Clotho
26.5R@1Cacophony
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CacophonyYear=2024, Setting=ZS2026.05 | 26.5 | — | — | — | — | |
| MGA-CLAPYear=2024, Setting=ZS2026.05 | 26.5 | — | — | — | — | |
| CacophonyYear=2024, Setting=ZS2026.05 | 26.5 | — | — | — | — | |
| MGA-CLAPYear=2024, Setting=ZS2026.05 | 26.5 | — | — | — | — | |
| WavCaps-CNN14Year=2023, Setting=ZS2026.05 | 25.9 | — | — | — | — | |
| WavCaps-CNN14Year=2023, Setting=ZS2026.05 | 25.9 | — | — | — | — | |
| LAION-CLAP fusionYear=2023, Setting=ZS2026.05 | 25.7 | — | — | — | — | |
| LAION-CLAP fusionYear=2023, Setting=ZS2026.05 | 25.7 | — | — | — | — | |
| FLAP fusionYear=2024, Setting=ZS2026.05 | 25.5 | — | — | — | — | |
| FLAP fusionYear=2024, Setting=ZS2026.05 | 25.5 | — | — | — | — | |
| M2D-CLAPYear=2025, Setting=ZS2026.05 | 25 | — | — | — | — | |
| M2D-CLAPYear=2025, Setting=ZS2026.05 | 25 | — | — | — | — | |
| WavCaps-HTSATYear=2023, Setting=ZS2026.05 | 23.4 | — | — | — | — | |
| WavCaps-HTSATYear=2023, Setting=ZS2026.05 | 23.4 | — | — | — | — | |
| Microsoft-CLAPModality=a2t2026.04 | 23.2 | 47.5 | 57.6 | — | 15.4 | |
| GLAPYear=2025, Setting=ZS2026.05 | 21.8 | — | — | — | — | |
| GLAPYear=2025, Setting=ZS2026.05 | 21.8 | — | — | — | — | |
| WavCaps-CNN14Year=2023, Setting=ZS2026.05 | 21.2 | — | — | — | — | |
| MGA-CLAPYear=2024, Setting=ZS2026.05 | 20.8 | — | — | — | — | |
| FLAP fusionYear=2024, Setting=ZS2026.05 | 20.3 | — | — | — | — | |
| CacophonyYear=2024, Setting=ZS2026.05 | 20.2 | — | — | — | — | |
| M2D-CLAPYear=2025, Setting=ZS2026.05 | 20.1 | — | — | — | — | |
| MS-CLAPYear=2023, Setting=ZS2026.05 | 20 | — | — | — | — | |
| MS-CLAPYear=2023, Setting=ZS2026.05 | 20 | — | — | — | — | |
| WavCaps-HTSATYear=2023, Setting=ZS2026.05 | 19.5 | — | — | — | — | |
| GLAPYear=2025, Setting=ZS2026.05 | 19.4 | — | — | — | — | |
| OmniRetriever-7BYear=2026, Setting=ZS2026.05 | 19.14 | — | — | — | — | |
| ProposedModality=a2t, audio encoder=HTSAT-tiny, text encoder=RoBERTa-large2026.04 | 18.3 | 48.2 | 59.1 | — | 16.2 | |
| LAION-CLAP fusionYear=2023, Setting=ZS2026.05 | 18.2 | — | — | — | — | |
| LAION-CLAPModality=a2t2026.04 | 17.5 | 37 | 45.5 | — | 15.5 | |
| MS-CLAPYear=2023, Setting=ZS2026.05 | 16.7 | — | — | — | — | |
| LanguageBind (CLIP-H/14)Year=2024, Setting=ZS2026.05 | 16.7 | — | — | — | — | |
| OmniRetriever-7BYear=2026, Setting=ZS2026.05 | 16.1 | — | — | — | — | |
| OmniRetriever-7BYear=2026, Setting=ZS2026.05 | 16.08 | — | — | — | — | |
| MMTYear=2022, Setting=FT2026.05 | 7 | — | — | — | — | |
| MMTYear=2022, Setting=FT2026.05 | 7 | — | — | — | — | |
| MMTYear=2022, Setting=FT2026.05 | 6.7 | — | — | — | — | |
| ImageBind (ViT-H)Year=2023, Setting=ZS2026.05 | 6 | — | — | — | — | |
| Gemini Embedding 2 (closed)Year=2026, Setting=ZS2026.05 | 5.19 | — | — | — | — | |
| SGD-CLIPTime=347.48s2026.04 | 3.73 | 12.44 | 19.23 | — | — | |
| Omni-Embed-NemotronYear=2025, Setting=ZS2026.05 | 3.5 | — | — | — | — | |
| UniConTime=13.45s2026.04 | 3.35 | 13.11 | 19.43 | — | — | |
| Gemini Embedding 2 (closed)Year=2026, Setting=ZS2026.05 | 1.34 | — | — | — | — | |
| Gemini Embedding 2Year=2026, Setting=ZS2026.05 | 1.3 | — | — | — | — | |
| ASK+Interaction Strategy=Local, Knowledge Source=WavCaps2025.12 | 0.195 | 0.414 | 0.545 | — | — | |
| ASK*Interaction Strategy=Local, Knowledge Source=Original training set2025.12 | 0.192 | 0.416 | 0.545 | — | — | |
| ASK†Interaction Strategy=Local, Knowledge Source=Gemini-annotated training set2025.12 | 0.191 | 0.419 | 0.539 | — | — | |
| Xie et al., 2024Interaction Strategy=Local2025.12 | 0.181 | 0.402 | 0.534 | — | — | |
| No alignment2026.04 | 0.1 | 0.48 | 0.96 | — | — | |
| Reference Model (Mei et al., 2024)2025.09 | — | — | — | 27.1 | — | |
| WAVE 7BModel Size=7B2025.09 | — | — | — | 24.5 | — |