Audio-Visual Segmentation on AVSBench MS3 (test)
65Jaccard Index (IoU)MUTR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MUTRBackbone=Swin-L2023.05 | 65 | 69 | 73 | |
| MUTRBackbone=Video-Swin-B2023.05 | 64 | 68.8 | 73.5 | |
| MUTRBackbone=PVT-V22023.05 | 63.7 | 67.9 | 72 | |
| MUTRBackbone=Video-Swin-S2023.05 | 62.7 | 67.3 | 71.8 | |
| MUTRBackbone=Video-Swin-T2023.05 | 59.2 | 64 | 68.7 | |
| MUTRBackbone=ResNet-1012023.05 | 59 | 63.7 | 68.3 | |
| AVSegFormerVisual Backbone=PVT-v22025.03 | 58.4 | — | 69.3 | |
| CrabVisual Backbone=ViT-L/142025.03 | 58.21 | — | 66.24 | |
| MoLTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=6.3, Total Params(M)=339.5, Memory (GB)=3.72025.11 | 57.1 | — | 70.3 | |
| MUTRBackbone=ResNet-502023.05 | 57 | 61.6 | 66.1 | |
| MoLTVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=8.3, Total Params(M)=313.1, Memory (GB)=3.62025.11 | 55.1 | — | 68.2 | |
| MettleVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=21.6, Total Params(M)=291.7, Memory (GB)=2.92025.11 | 55.1 | — | — | |
| AVMoEVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=54.4, Total Params(M)=501.2, Memory (GB)=9.52025.11 | 54.5 | — | 68.7 | |
| BaselineBackbone=PVT-V22023.05 | 54 | 59.3 | 64.5 | |
| TPAVIVisual Backbone=PVT-v22025.03 | 54 | — | 64.5 | |
| AVSBenchVisual Backbone=PVT-v22025.03 | 54 | — | 23.2 | |
| AVSVisual Encoder=PVT-V2, Audio Encoder=VGGish, Trainable Params(%)=58.7, Total Params(M)=174.5, Memory (GB)=4.82025.11 | 54 | — | 64.5 | |
| MettleVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=11.1, Total Params(M)=257.2, Memory (GB)=2.72025.11 | 53.7 | — | — | |
| DG-SCTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=61.5, Total Params(M)=594.8, Memory (GB)=92025.11 | 53.5 | — | 64.2 | |
| CATRVisual Backbone=ResNet-502025.03 | 52.8 | — | 65.3 | |
| BAVSVisual Backbone=ResNet-502025.03 | 50.2 | — | 62.4 | |
| LAVisHVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=14, Total Params(M)=266.4, Memory (GB)=5.92025.11 | 49.8 | — | 60.3 | |
| LAVisHVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=47.6, Total Params(M)=389.72025.11 | 49.1 | — | 59.9 | |
| BaselineBackbone=ResNet-502023.05 | 47.9 | 52.9 | 57.8 | |
| TPAVIVisual Backbone=ResNet-502025.03 | 47.9 | — | 57.8 | |
| iGANVisual Backbone=Swin-T2025.03 | 42.9 | — | 54.4 | |
| SSTBackbone=ResNet-502023.05 | 42.6 | 49.9 | 57.2 | |
| ACL-SSLObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 41.08 | — | 46.67 | |
| LGVTBackbone=Swin-B2023.05 | 40.7 | 50 | 59.3 | |
| LGVTVisual Backbone=Swin-T2025.03 | 40.7 | — | 59.3 | |
| SOUPLEObject-guided refinement (OGL)=false, Zero-shot evaluation=true, Training epochs=502026.03 | 40.19 | — | 46 | |
| SOUPLEObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 38.96 | — | 43.3 | |
| LVSBackbone=ResNet-182023.05 | 29.5 | 31.3 | 33 | |
| MarginNCEObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 27.31 | — | 31.56 | |
| AudioCLIPObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 27.06 | — | 26.48 | |
| WAV2CLIPObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 25.09 | — | 23.84 | |
| SLAVCObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 24.37 | — | 25.56 | |
| FNACObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 21.98 | — | 22.5 |