Audio Captioning Evaluation on BRACE Main 1.0
67.63AudioCaps-Main HH ScoreCAF-Score
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=Qwen3-Omni-Instruct, Backbone CLAP=M2D-CLAP2026.03 | 67.63 | 86.97 | 67.91 | 75.28 | 70.66 | 77.8 | 66.52 | 71.28 | 73.11 | |
| CLAIR-AReference Protocol=Reference-based2026.03 | 66.19 | 90.79 | 67.74 | 76.51 | 56.29 | 91.94 | 65.63 | 74.39 | 75.36 | |
| CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=M2D-CLAP2026.03 | 65.47 | 80.22 | 63.99 | 70.48 | 67.66 | 66.21 | 62.81 | 64.69 | 67.35 | |
| S-CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=M2D-CLAP2026.03 | 65.47 | 80.22 | 63.99 | 70.48 | 70.06 | 69.94 | 63.85 | 66.91 | 68.59 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=AudioFlamingo3-Base, Backbone CLAP=MGA-CLAP2026.03 | 65.47 | 71.01 | 67.91 | 68.82 | 70.06 | 70.92 | 63.26 | 66.99 | 67.83 | |
| CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=MS-CLAP2026.03 | 64.03 | 48.76 | 55.08 | 53.71 | 67.07 | 70.92 | 65.93 | 67.95 | 61.42 | |
| S-CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=MS-CLAP2026.03 | 64.03 | 46.07 | 53.65 | 51.97 | 65.87 | 72.89 | 67.26 | 69.21 | 61.3 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=AudioFlamingo3-Base, Backbone CLAP=M2D-CLAP2026.03 | 64.03 | 70.34 | 67.38 | 68.12 | 67.66 | 71.71 | 63.7 | 67.21 | 67.63 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=Qwen3-Omni-Instruct, Backbone CLAP=MS-CLAP2026.03 | 64.03 | 60.67 | 63.99 | 62.71 | 67.66 | 79.17 | 70.07 | 73.21 | 68.39 | |
| CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=LAION-CLAP2026.03 | 63.31 | 77.3 | 65.42 | 69.78 | 59.28 | 75.64 | 65.63 | 68.62 | 69.15 | |
| S-CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=LAION-CLAP2026.03 | 63.31 | 77.3 | 65.42 | 69.78 | 59.88 | 75.83 | 66.96 | 69.43 | 69.59 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=AudioFlamingo3-Base, Backbone CLAP=LAION-CLAP2026.03 | 63.31 | 76.63 | 67.38 | 70.48 | 58.68 | 77.01 | 67.85 | 70.17 | 70.31 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=Qwen3-Omni-Instruct, Backbone CLAP=LAION-CLAP2026.03 | 63.31 | 81.57 | 68.09 | 72.75 | 61.68 | 80.16 | 68 | 71.8 | 72.23 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=Qwen3-Omni-Instruct, Backbone CLAP=MGA-CLAP2026.03 | 63.31 | 81.12 | 67.74 | 72.4 | 73.65 | 75.44 | 63.85 | 69.43 | 70.79 | |
| CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=MGA-CLAP2026.03 | 62.59 | 73.48 | 65.24 | 68.12 | 71.26 | 67.58 | 63.41 | 65.95 | 66.95 | |
| S-CLAPScoreMetric Architecture=CLAP-ONLY, Reference Protocol=Reference-free, Backbone=MGA-CLAP2026.03 | 62.59 | 73.48 | 65.24 | 68.12 | 72.46 | 68.96 | 62.52 | 66.17 | 67.51 | |
| CAF-ScoreMetric Architecture=CLAP + LALM, Reference Protocol=Reference-free, Backbone LALM=AudioFlamingo3-Base, Backbone CLAP=MS-CLAP2026.03 | 62.59 | 48.09 | 60.07 | 55.72 | 64.67 | 73.87 | 68.74 | 70.17 | 63.54 | |
| FENSEReference Protocol=Reference-based2026.03 | 61.15 | 84.49 | 67.38 | 73.28 | 56.89 | 84.68 | 64.15 | 70.98 | 72.04 | |
| FLEURMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=AudioFlamingo3-Base2026.03 | 53.24 | 52.58 | 66.49 | 59.48 | 63.47 | 68.37 | 61.19 | 64.17 | 62.02 | |
| FLEURMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=Qwen3-Omni-Instruct2026.03 | 53.24 | 78.65 | 66.31 | 69.52 | 67.66 | 74.66 | 61.63 | 67.28 | 68.31 | |
| FLEURMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=Qwen3-Omni-Thinking2026.03 | 51.8 | 71.46 | 58.82 | 62.88 | 58.68 | 68.96 | 56.74 | 61.58 | 62.18 | |
| FLEURMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=AudioFlamingo3-Think2026.03 | 41.73 | 52.58 | 56.51 | 53.19 | 59.28 | 54.81 | 55.11 | 55.11 | 54.45 | |
| Raw ScoreMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=AudioFlamingo3-Base2026.03 | 33.81 | 28.09 | 35.12 | 32.23 | 13.77 | 22.4 | 23.26 | 21.76 | 26.56 | |
| Raw ScoreMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=Qwen3-Omni-Thinking2026.03 | 25.9 | 55.73 | 43.14 | 45.94 | 41.92 | 55.4 | 42.22 | 47.15 | 46.59 | |
| Raw ScoreMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=Qwen3-Omni-Instruct2026.03 | 16.55 | 48.54 | 41 | 40.96 | 35.93 | 54.62 | 45.04 | 47.52 | 44.51 | |
| Raw ScoreMetric Architecture=LALM-ONLY, Reference Protocol=Reference-free, Backbone=AudioFlamingo3-Think2026.03 | 15.83 | 22.47 | 19.07 | 20 | 21.56 | 21.22 | 21.33 | 21.32 | 20.71 |