Image-based Audio-Visual Segmentation Quality Assessment on MQ-RAVSBench (Unseen)
0.192RMSE (Perfect)Video-LLaMA3-7B
Evaluation Results
| Method | Links | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Video-LLaMA3-7BEvaluation Protocol=Image-based2026.02 | 0.192 | 0.259 | 0.306 | 0.264 | 0.298 | 0.254 | 0.289 | 0.524 | 0.951 | 0.371 | 88.28 | 82 | 0 | 0 | 0 | 0 | 0 | 0 | 16 | 9.92 | 94.09 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 16 | 10.47 | |
| MQ-AuditorEvaluation Protocol=Image-based2026.02 | 0.274 | 0.161 | 0.135 | 0.203 | 0.154 | 0.257 | 0.269 | 0.31 | 0.508 | 0.252 | 64.34 | 9,329 | 97.07 | 76.22 | 89.43 | 27.3 | 41.26 | 77.73 | 7,092 | 70.84 | 64.34 | 47.76 | 76.82 | 37.05 | 81.2 | 28.77 | 30.6 | 44.57 | 7,441 | 53.95 | |
| Gemini-3-Flash-PreviewEvaluation Protocol=Image-based2026.02 | 0.409 | 0.357 | 0.334 | 0.32 | 0.298 | 0.345 | 0.314 | 0.301 | 0.25 | 0.325 | 18.39 | 7,807 | 79.4 | 0.86 | 3.16 | 3.14 | 3.97 | 73.84 | 8,042 | 37.92 | 21.98 | 19.78 | 64.31 | 23.29 | 37.47 | 24.85 | 47.15 | 69.08 | 8,849 | 44.04 | |
| Qwen2.5-Omni-7BEvaluation Protocol=Image-based2026.02 | 0.418 | 0.501 | 0.455 | 0.431 | 0.412 | 0.394 | 0.37 | 0.501 | 0.916 | 0.489 | 74.74 | 2,471 | 26.19 | 0 | 0 | 0 | 0 | 1.13 | 371 | 14.5 | 74.44 | 25.08 | 0 | 14.81 | 0 | 13.64 | 0 | 6.94 | 371 | 15.4 | |
| Ming-Flash-OmniEvaluation Protocol=Image-based2026.02 | 0.981 | 0.886 | 0.843 | 0.85 | 0.757 | 0.86 | 0.761 | 0.653 | 0.518 | 0.79 | 5.52 | 1,585 | 14.6 | 0 | 0 | 0.43 | 1.71 | 26.63 | 1,764 | 9.15 | 5.94 | 1.36 | 11.34 | 2.99 | 9.69 | 2.99 | 8.86 | 57.88 | 6,620 | 18.58 |