Audio-Visual Segmentation on AVSBench S4
76.6MJ ScoreSelM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SelMVisual Backbone=R50, Audio Backbone=VGGish, GPU Latency (ms)=25.3, Mobile Latency (ms)=1003.8, Params=117.6M2026.05 | 76.6 | — | 86.2 | |
| AVSegFormerVisual Backbone=R50, Audio Backbone=VGGish, GPU Latency (ms)=29.0, Mobile Latency (ms)=1271.4, Params=151.1M2026.05 | 76.5 | — | 85.9 | |
| LightAVSegVisual Backbone=Sea, Audio Backbone=MNetV2, GPU Latency (ms)=15.9, Mobile Latency (ms)=163.4, Params=20.5M2026.05 | 75.6 | — | 86.2 | |
| AVSBenchVisual Backbone=R50, Audio Backbone=VGGish, GPU Latency (ms)=21.2, Mobile Latency (ms)=753.5, Params=91.4M2026.05 | 72.8 | — | 84.8 | |
| SelMVisual Backbone=Sea, Audio Backbone=MNetV2, GPU Latency (ms)=18.7, Mobile Latency (ms)=308.6, Params=39.5M2026.05 | 59.1 | — | 77.4 | |
| AVSegFormerVisual Backbone=Sea, Audio Backbone=MNetV2, GPU Latency (ms)=22.2, Mobile Latency (ms)=432.6, Params=51.0M2026.05 | 53.8 | — | 71.4 | |
| AVSBenchVisual Backbone=Sea, Audio Backbone=MNetV2, GPU Latency (ms)=19.5, Mobile Latency (ms)=237.1, Params=30.2M2026.05 | 47.9 | — | 64.5 | |
| AVSVisual Encoder=PVT-V2, Audio Encoder=VGGish, Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=102.4, Total Params (M)=174.52022.12 | — | 78.7 | — | |
| AVS*Visual Encoder=Swin-V2-L, Audio Encoder=VGGish, Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=249.7, Total Params (M)=321.82022.12 | — | 80.4 | — | |
| LAVISHVisual Encoder=Swin-V2-L, Audio Encoder=Swin-V2-L (shared), Visual Pretrain Dataset=ImageNet, Trainable Params (M)=37.2, Total Params (M)=266.42022.12 | — | 80.1 | — | |
| LVS+Visual Encoder=ResNet18, Audio Encoder=ResNet18, Visual Pretrain Dataset=ImageNet2022.12 | — | 37.9 | — | |
| MMSL+Visual Encoder=ResNet-18, Audio Encoder=CRNN, Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet2022.12 | — | 44.9 | — |