Audio Captioning on AudioCaps
80.3CIDErEnCLAP-large
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| EnCLAP-large2024.01 | 80.3 | 25.5 | 18.8 | 49.5 | — | — | — | — | |
| CoDi2023.05 | 78.9 | — | 18.2 | 48 | — | — | — | — | |
| CoDi2023.12 | 78.9 | — | — | — | — | — | — | — | |
| HTSAT-BART2025.03 | 78.7 | 25 | 18.2 | 48.5 | — | — | — | — | |
| HTSAT-BART2023.05 | 78.7 | 25 | — | — | — | 28.3 | 50.7 | — | |
| Method [18]2023.05 | 78.7 | — | — | — | — | — | — | — | |
| WavCaps2024.01 | 78.7 | 25 | 18.2 | 48.5 | — | — | — | — | |
| SOTA2024.06 | 78.7 | — | — | — | — | — | — | — | |
| VAST2023.05 | 78.2 | — | — | — | — | — | — | — | |
| VAST2023.05 | 78.1 | 24.7 | — | — | — | 29.5 | 50.9 | — | |
| EnCLAP-base2024.01 | 78 | 24.7 | 18.6 | 48.3 | — | — | — | — | |
| AL-MixGen2024.01 | 76.9 | 24.2 | 18.1 | 47.5 | — | — | — | — | |
| CNeXt-Trans2024.01 | 76 | — | 18.2 | 47.1 | — | — | — | — | |
| CNN14-BART2025.03 | 75.6 | 24.7 | 17.9 | 46.8 | — | — | — | — | |
| CNN14-BART2023.05 | 75.6 | 24.7 | — | — | — | 27.2 | 49.9 | — | |
| AL-MixGen2023.05 | 75.5 | — | 17.7 | — | — | — | — | — | |
| BART-Finetune2023.05 | 75.3 | — | 16.6 | 46 | — | — | — | — | |
| BART-tags2024.01 | 75.3 | 24.1 | 17.6 | 46.5 | — | — | — | — | |
| Pengi2024.01 | 75.2 | 23.2 | 18.2 | 46.7 | — | — | — | — | |
| VALOR2023.05 | 74.1 | — | — | — | — | — | — | — | |
| VALOR-B2023.05 | 74.1 | 23.1 | — | — | — | 27 | 49.4 | — | |
| VALOR_B2023.04 | 74.1 | 23.1 | — | — | — | 27 | 49.4 | — | |
| AF-Next-Instruct2026.04 | 74 | — | — | — | — | — | — | — | |
| CNN14-GPT22024.01 | 73.3 | 24 | 17.7 | 45.5 | — | — | — | — | |
| Cacophony2025.03 | 72.8 | 23.6 | 16.8 | 44.8 | — | — | — | — | |
| M2D-CLAP2025Audio encoder AudioSet fine-tuning=true2025.03 | 72.4 | 24.3 | 17.6 | 45 | 63.9 | — | — | — | |
| LAION-CLAPAudio encoder AudioSet fine-tuning=true2025.03 | 71.7 | 23.8 | 17.6 | 44.6 | 64 | — | — | — | |
| Audio Flamingo 32026.04 | 70 | — | — | — | — | — | — | — | |
| M2D-CLAP stage2Audio encoder AudioSet fine-tuning=false2025.03 | 69.6 | 23.7 | 17.2 | 43.4 | 62.5 | — | — | — | |
| M2D-CLAP stage2.1Audio encoder AudioSet fine-tuning=false2025.03 | 69.4 | 23.6 | 17.1 | 43.2 | 62.3 | — | — | — | |
| M2D-CLAP stage1Audio encoder AudioSet fine-tuning=false2025.03 | 68.3 | 23.3 | 16.9 | 42.6 | 61.9 | — | — | — | |
| LHDFF2024.01 | 68 | 23.2 | 17.1 | 42.6 | — | — | — | — | |
| ACT2024.01 | 67.9 | 22.2 | 16 | 42 | — | — | — | — | |
| ACT2023.04 | 67.9 | 22.2 | — | — | — | 25.2 | 46.8 | — | |
| Liu et al.2023.04 | 66.7 | 23.2 | — | — | — | 25.1 | 48 | — | |
| MiCo2024.06 | 66.2 | — | — | — | — | — | — | — | |
| Xu et al.2023.05 | 66 | 22.9 | — | — | — | 23.1 | 46.7 | — | |
| Xu et al. [115]2023.04 | 66 | 22.9 | — | — | — | 23.1 | 46.7 | — | |
| CLAP2023Audio encoder AudioSet fine-tuning=true2025.03 | 65.2 | 22.4 | 16.3 | 40.7 | 60.5 | — | — | — | |
| AudioCaps2023.05 | 59.3 | — | 14.4 | 36.9 | — | — | — | — | |
| Kim et al.2023.04 | 59.3 | 20.3 | — | — | — | 21.9 | 45 | — | |
| Audio Flamingo# Shots=162024.02 | 54.6 | — | — | — | — | — | — | — | |
| Audio Flamingo# Shots=82024.02 | 53.8 | — | — | — | — | — | — | — | |
| Audio Flamingo# Shots=42024.02 | 51.8 | — | — | — | — | — | — | — | |
| LTBM (local)Keep Ratio=0.752026.05 | 49.51 | — | — | — | — | — | — | — | |
| Global Top-KKeep Ratio=0.752026.05 | 49.12 | — | — | — | — | — | — | — | |
| UIO-2XXLModel Size=XXL2023.12 | 48.9 | — | — | — | — | — | — | — | |
| BaselineKeep Ratio=0.752026.05 | 48.25 | — | — | — | — | — | — | — | |
| BaselineKeep Ratio=0.52026.05 | 48.25 | — | — | — | — | — | — | — | |
| BaselineKeep Ratio=0.252026.05 | 48.25 | — | — | — | — | — | — | — | |
| Global MergeKeep Ratio=0.752026.05 | 48.23 | — | — | — | — | — | — | — | |
| Global MergeKeep Ratio=0.52026.05 | 48.05 | — | — | — | — | — | — | — | |
| LTBM (local)Keep Ratio=0.52026.05 | 47.82 | — | — | — | — | — | — | — | |
| Segmentwise Top-KKeep Ratio=0.752026.05 | 47.74 | — | — | — | — | — | — | — | |
| UniAvgKeep Ratio=0.52026.05 | 47.48 | — | — | — | — | — | — | — | |
| Segmentwise Top-KKeep Ratio=0.52026.05 | 47.28 | — | — | — | — | — | — | — | |
| Global Top-KKeep Ratio=0.52026.05 | 45.78 | — | — | — | — | — | — | — | |
| UIO-2LModel Size=L2023.12 | 45.7 | — | — | — | — | — | — | — | |
| UIO-2XLModel Size=XL2023.12 | 45.7 | — | — | — | — | — | — | — | |
| LTBM (local)Keep Ratio=0.252026.05 | 39.09 | — | — | — | — | — | — | — | |
| Global MergeKeep Ratio=0.252026.05 | 38.09 | — | — | — | — | — | — | — | |
| RECAP# Shots=42024.02 | 35.9 | — | — | — | — | — | — | — | |
| Segmentwise Top-KKeep Ratio=0.252026.05 | 32.67 | — | — | — | — | — | — | — | |
| Global Top-KKeep Ratio=0.252026.05 | 29.57 | — | — | — | — | — | — | — | |
| ImageBindZero-shot=true2023.12 | 27.8 | — | — | — | — | — | — | — | |
| UniAvgKeep Ratio=0.252026.05 | 27.54 | — | — | — | — | — | — | — | |
| Captioning-scratchLabel=Caption, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 43.58 | |
| Captioning-scratch (Ours)Label=Caption*, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 45.07 | |
| CLAP2023 (Ref)2025.03 | — | — | — | 45.5 | — | — | — | — | |
| Contrastive-scratchLabel=Caption, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 44.5 | |
| Contrastive-scratch (Ours)Label=Caption*, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 44.54 | |
| Kimi-Audio-InstructDecoder Size=7B2025.08 | — | — | — | — | 49 | — | — | — | |
| MiDashengLMDecoder Size=7B2025.08 | — | — | — | — | 62.18 | — | — | — | |
| MTC (AudioSet)Label=Tag, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 46.67 | |
| MTC (Ours-UTS)Label=Tag*, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 44.4 | |
| Multi-Task (Ours)Label=Tag*, Caption*, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 45.17 | |
| PAR (Ours-UTS)Label=Tag Sequence*, Evaluation Protocol=audio-language alignment2026.03 | — | — | — | — | — | — | — | 44.8 | |
| Pengi2023.05 | — | — | — | 46.67 | — | — | — | — | |
| Qwen2.5-OmniDecoder Size=7B2025.08 | — | — | — | — | 60.79 | — | — | — |