Text-to-audio retrieval on AudioCaps
55.2Recall@1InternVideo2-6B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| InternVideo2-6BEvaluation protocol=Finetuning2024.03 | 55.2 | — | — | — | |
| VASTEvaluation protocol=Finetuning2024.03 | 52 | — | — | — | |
| PEAV LA-Enc Params.=1.1B, Data (M)=124, Sampling=30 FPS2025.12 | 45.8 | — | — | — | |
| PEAV LA-Enc Params.=1.1B, Data (M)=124, Sampling=16 Frames2025.12 | 44.7 | — | — | — | |
| PEAV L-OODA-Enc Params.=1.1B, Data (M)=114, Sampling=30 FPS, OOD=true2025.12 | 43.4 | — | — | — | |
| PEAV BA-Enc Params.=.2B, Data (M)=124, Sampling=16 Frames2025.12 | 43.1 | — | — | — | |
| PEAV BA-Enc Params.=0.2B, Data (M)=124, Sampling=30 FPS2025.12 | 42.7 | — | — | — | |
| PEAV SA-Enc Params.=.09B, Data (M)=124, Sampling=30 FPS2025.12 | 41.8 | — | — | — | |
| PEAV SA-Enc Params.=.09B, Data (M)=124, Sampling=16 Frames2025.12 | 41.2 | — | — | — | |
| MTC (AudioSet)Label=Tag, Evaluation Protocol=audio-language alignment2026.03 | 40.46 | — | — | — | |
| InternVideo2-6BEvaluation protocol=Zero-shot2024.03 | 37.1 | — | — | — | |
| PMRLSetting=Zero-shot2025.07 | 36.1 | 75.9 | — | — | |
| CLAP-FusionA-Enc Params.=.03B, Data (M)=32025.12 | 35.4 | — | — | — | |
| ProposedModality=t2a, audio encoder=HTSAT-tiny, text encoder=RoBERTa-large2026.04 | 35.2 | 84.4 | 71.5 | 52.1 | |
| Proposed audio-text retrieval framework2026.04 | 35.2 | 84.4 | 71.5 | 52.1 | |
| GRAMSetting=Zero-shot2025.07 | 34.6 | 77.4 | — | — | |
| LAION-CLAPModality=t2a2026.04 | 34.1 | 82.7 | 69.7 | 49 | |
| PEAV L (PT)A-Enc Params.=1.1B, Data (M)=92, Sampling=30 FPS, PT=true2025.12 | 33.7 | — | — | — | |
| VASTSetting=Zero-shot2025.07 | 33.7 | 77.1 | — | — | |
| CLAPA-Enc Params.=.03B, Data (M)=32025.12 | 31.6 | — | — | — | |
| AFlamingo2A-Enc Params.=0.3B, Data (M)=82025.12 | 29.8 | — | — | — | |
| Contrastive-scratch (Ours)Label=Caption*, Evaluation Protocol=audio-language alignment2026.03 | 29.66 | — | — | — | |
| Microsoft-CLAPModality=t2a2026.04 | 28.9 | 77 | 63 | 43.2 | |
| Contrastive-scratchLabel=Caption, Evaluation Protocol=audio-language alignment2026.03 | 28.73 | — | — | — | |
| VIP-ANTEvaluation protocol=Zero-shot2024.03 | 27.7 | — | — | — | |
| VIP-ANTSetting=Zero-shot2025.07 | 27.7 | 37.7 | — | — | |
| M2D-CLAPA-Enc Params.=.09B, Data (M)=22025.12 | 27.4 | — | — | — | |
| Multi-Task (Ours)Label=Tag*, Caption*, Evaluation Protocol=audio-language alignment2026.03 | 27.07 | — | — | — | |
| PAR (Ours-UTS)Label=Tag Sequence*, Evaluation Protocol=audio-language alignment2026.03 | 26.76 | — | — | — | |
| Captioning-scratch (Ours)Label=Caption*, Evaluation Protocol=audio-language alignment2026.03 | 26.24 | — | — | — | |
| MTC (Ours-UTS)Label=Tag*, Evaluation Protocol=audio-language alignment2026.03 | 26.14 | — | — | — | |
| Captioning-scratchLabel=Caption, Evaluation Protocol=audio-language alignment2026.03 | 26.03 | — | — | — | |
| GeminiRetrieval=BM252026.04 | 25.6 | 73 | 60.1 | 40.2 | |
| AudioGPTRetrieval=BM252026.04 | 25.5 | 72.7 | 60.9 | 41.7 | |
| MS-CLAP23’A-Enc Params.=.08B, Data (M)=0.12025.12 | 23.4 | — | — | — | |
| GeminiRetrieval=Semantic2026.04 | 21.8 | 50.4 | 48.1 | 31.7 | |
| LTURetrieval=BM252026.04 | 21.1 | 60.3 | 57.3 | 30.8 | |
| PengiRetrieval=BM252026.04 | 20.9 | 53.2 | 50.3 | 23.9 | |
| GAMARetrieval=BM252026.04 | 20.8 | 59.7 | 52 | 29.4 | |
| LangBindA-Enc Params.=0.3B, Data (M)=102025.12 | 19.7 | — | — | — | |
| LanguageBindSetting=Zero-shot2025.07 | 19.7 | 67.6 | — | — | |
| SalmonnRetrieval=BM252026.04 | 19.5 | 59.6 | 48.1 | 22.8 | |
| GeminiRetrieval=Lexical2026.04 | 19.3 | 42.8 | 39.3 | 24.1 | |
| LTURetrieval=Semantic2026.04 | 18.9 | 46.7 | 39.5 | 21.8 | |
| AudioGPTRetrieval=Semantic2026.04 | 18.4 | 48.5 | 47.2 | 28.8 | |
| SalmonnRetrieval=Semantic2026.04 | 17.9 | 47.2 | 38.6 | 20 | |
| SalmonnRetrieval=Lexical2026.04 | 17.6 | 39.2 | 29.4 | 14.7 | |
| PengiRetrieval=Semantic2026.04 | 17.3 | 39.9 | 35.2 | 18.6 | |
| LTURetrieval=Lexical2026.04 | 17.2 | 42.2 | 30.3 | 20.1 | |
| AudioGPTRetrieval=Lexical2026.04 | 16.5 | 39.2 | 38.7 | 21.9 | |
| PengiRetrieval=Lexical2026.04 | 12.8 | 36.2 | 30.7 | 18.4 | |
| GAMARetrieval=Semantic2026.04 | 12.8 | 40.8 | 39.3 | 19.9 | |
| AVFICSetting=Zero-shot, Variant=22025.07 | 10.6 | 45.2 | — | — | |
| GAMARetrieval=Lexical2026.04 | 10.5 | 33.8 | 31.4 | 15.8 | |
| ImageBindSetting=Zero-shot2025.07 | 9.3 | 42.3 | — | — | |
| AVFICSetting=Zero-shot, Variant=12025.07 | 8.7 | 37.7 | — | — | |
| ImageBindA-Enc Params.=.09B, Data (M)=32025.12 | 6.6 | — | — | — |