Audio Captioning on Clotho (test)
19.7METEORHYU (ensemble)
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HYU (ensemble)Setting=Supervised2023.11 | 19.7 | 20.2 | 42.2 | 54.1 | 14.6 | 34.3 | — | — | — | — | — | |
| SLAM-AAC2026.01 | 19.7 | — | — | 51.5 | 14.8 | — | — | — | — | — | — | |
| t100→a100Pre-training=WavCaps+SoundVECaps, Beam size=5, Embedding Dimension=1002026.05 | 18.5 | 15.3 | 37.7 | 41.6 | 13.3 | — | — | — | — | 27.4 | — | |
| HTSAT-BARTSetting=Supervised2023.11 | 18.4 | 16.8 | 38.3 | 46.2 | 13.3 | 29.7 | — | — | — | — | — | |
| t100→a100Pre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=5, Embedding Dimension=1002026.05 | 18.3 | 15.4 | 37.8 | 41.8 | 13.3 | — | — | — | — | 27.5 | — | |
| SLAM-LLMScenario=In Domain, Zero-shot=true2026.01 | 18.2 | — | — | 43.8 | 13.3 | — | — | — | — | 28.5 | 53 | |
| DRCapPre-training=WavCaps+SoundVECaps2026.05 | 18.2 | — | — | 43.8 | 13.3 | — | — | — | — | 28.5 | — | |
| SLAP2026.01 | 18.1 | — | — | 43.7 | 13.1 | — | — | — | — | — | — | |
| t→a PDPre-training=WavCaps+SoundVECaps, Beam size=52026.05 | 18.1 | 15.6 | 37.9 | 43.1 | 13.2 | — | — | — | — | 28.2 | — | |
| a→aPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 18 | 16.6 | 38 | 41.1 | 12.6 | — | — | — | — | 26.8 | — | |
| t→a PDPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 17.9 | 15.1 | 37.5 | 42.3 | 13 | — | — | — | — | 27.7 | — | |
| a100→a100Pre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=5, Embedding Dimension=1002026.05 | 17.9 | 16.5 | 37.6 | 42 | 12.4 | — | — | — | — | 27.2 | — | |
| BLAT2026.01 | 17.8 | — | — | 41.5 | 12.6 | — | — | — | — | — | — | |
| M2D22026.01 | 17.8 | — | — | 43.5 | 12.4 | — | — | — | — | — | — | |
| a→aPre-training=WavCaps+SoundVECaps, Beam size=52026.05 | 17.7 | 16.3 | 37.9 | 41.6 | 12.4 | — | — | — | — | 27 | — | |
| a100→a100Pre-training=WavCaps+SoundVECaps, Beam size=5, Embedding Dimension=1002026.05 | 17.7 | 16.3 | 37.5 | 41 | 12.3 | — | — | — | — | 26.7 | — | |
| Zhang et al.Scenario=In Domain, Zero-shot=true2026.01 | 17.5 | — | — | 41.1 | 12.2 | — | — | — | — | 26.7 | 48.8 | |
| WSACScenario=In Domain, Zero-shot=true2026.01 | 17.4 | — | — | 37.1 | 12.3 | — | — | — | — | 24.7 | — | |
| t100rec→a100recPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 17.4 | 14.1 | 36.4 | 36 | 12.1 | — | — | — | — | 24 | — | |
| SoftHardPre-training=WavCaps, Backbone=HTSAT-BERT-ZS2026.05 | 17.3 | 15.6 | 37.5 | 40.3 | 11.9 | — | — | — | — | 26.1 | — | |
| t→a NNDPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 17.2 | 13.1 | 35.7 | 36 | 12.3 | — | — | — | — | 24.1 | — | |
| WSACPre-training=WavCaps, Backbone=HTSAT-BERT-ZS2026.05 | 16.9 | 12.6 | 35.9 | 35.7 | 11.8 | — | — | — | — | 23.8 | — | |
| t→a ESPre-training=WavCaps+SoundVECaps, Beam size=52026.05 | 16.6 | 12.3 | 34.9 | 32.5 | 11.2 | — | — | — | — | 21.9 | — | |
| t→a ESPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 16.3 | 11.6 | 34.3 | 30.8 | 10.6 | — | — | — | — | 20.7 | — | |
| t→a NIPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 15.5 | 12.5 | 33.5 | 27.4 | 10.3 | — | — | — | — | 18.8 | — | |
| Cacophony2026.01 | 15.3 | — | — | 41.5 | 10.6 | — | — | — | — | — | — | |
| SLAM-LLMScenario=Cross Domain, Zero-shot=true2026.01 | 15 | — | — | 33.3 | 10.4 | — | — | — | — | 21.8 | 52.2 | |
| t→a ADPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 14.7 | 11.1 | 33.3 | 26.6 | 8.6 | — | — | — | — | 17.6 | — | |
| t→a ADPre-training=WavCaps+SoundVECaps, Beam size=52026.05 | 14.6 | 9 | 30.7 | 24 | 8.6 | — | — | — | — | 16.3 | — | |
| Zhang et al.Scenario=Cross Domain, Zero-shot=true2026.01 | 13.2 | — | — | 24.8 | 9.3 | — | — | — | — | 17.1 | — | |
| WSACScenario=Cross Domain, Zero-shot=true2026.01 | 12 | — | — | 20.6 | 8.2 | — | — | — | — | 14.4 | — | |
| t-924→t-924Pre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 11.4 | 7.1 | 27.1 | 12 | 5.7 | — | — | — | — | 8.8 | — | |
| a-924→a-924Pre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 11 | 6.2 | 27.2 | 7.5 | 5 | — | — | — | — | 6.3 | — | |
| t-924→a-924Pre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 10.9 | 5.9 | 26 | 8.8 | 4.9 | — | — | — | — | 6.9 | — | |
| ZerAuCapSetting=Zero-shot, LLM=OPT2023.11 | 9.4 | 2.9 | 25.4 | 14 | 5.3 | 9.7 | — | — | — | — | — | |
| ZerAuCapScenario=not specified, Zero-shot=true2026.01 | 9.4 | — | — | 14 | 5.3 | — | — | — | — | 9.7 | — | |
| no_condPre-training=WavCaps, Backbone=HTSAT-BERT-ZS, Beam size=52026.05 | 8.5 | 3.5 | 24.9 | 5.4 | 2.3 | — | — | — | — | 3.9 | — | |
| No audio (baseline)Setting=Zero-shot2023.11 | 3.8 | 0 | 16.6 | 0.2 | 0.1 | 0.2 | — | — | — | — | — | |
| Pengi2023.05 | 0.172 | 0.15 | 0.375 | 0.416 | 0.126 | — | 0.57 | 0.369 | 0.242 | 0.271 | — | |
| Kim et al.training=supervised2023.05 | 0.159 | 0.142 | 0.366 | 0.319 | 0.111 | — | 0.539 | 0.346 | 0.227 | 0.215 | — | |
| Mei et al.training=supervised2023.05 | 0.157 | 0.127 | 0.351 | 0.313 | 0.105 | — | 0.516 | 0.318 | 0.204 | 0.209 | — | |
| Chen et al.training=supervised2023.05 | 0.153 | 0.141 | 0.35 | 0.314 | 0.102 | — | 0.516 | 0.325 | 0.215 | 0.208 | — | |
| Gontier et al.training=supervised2023.05 | 0.136 | 0.117 | 0.318 | 0.251 | 0.083 | — | 0.461 | 0.282 | 0.182 | 0.167 | — | |
| DAC-RLDDecoding=Nucleus (p = 0.95), Beam=12025.07 | — | 0.15 | — | — | — | — | — | — | — | 0.244 | — | |
| DAC-RLDDecoding=BS, Beam=52025.07 | — | 0.215 | — | — | — | — | — | — | — | 0.287 | — | |
| LoRA-MCLTraining variant=annealed, Decoding=Nucleus (p = 0.95), Beam=12025.07 | — | 0.098 | — | — | — | — | — | — | — | 0.325 | — | |
| LoRA-MCLTraining variant=ε = 0.05, Decoding=BS, Beam=52025.07 | — | 0.478 | — | — | — | — | — | — | — | 0.434 | — | |
| LoRA-MCLTraining variant=ρ = 0.999, Decoding=BS, Beam=52025.07 | — | 0.478 | — | — | — | — | — | — | — | 0.443 | — |