Audio-Visual Segmentation on AVSBench S4 (test)
89.8FMUTR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MUTRBackbone=Swin-L2023.05 | 89.8 | 85.7 | 81.5 | — | — | — | |
| MUTRBackbone=Video-Swin-B2023.05 | 89.7 | 85.7 | 81.6 | — | — | — | |
| MUTRBackbone=PVT-V22023.05 | 89.5 | 85.1 | 80.7 | — | — | — | |
| MUTRBackbone=Video-Swin-S2023.05 | 88.3 | 84.1 | 79.8 | — | — | — | |
| BaselineBackbone=PVT-V22023.05 | 87.9 | 83.3 | 78.7 | — | — | — | |
| MUTRBackbone=ResNet-1012023.05 | 87.6 | 83.1 | 78.5 | — | — | — | |
| LGVTBackbone=Swin-B2023.05 | 87.3 | 81.1 | 74.9 | — | — | — | |
| MUTRBackbone=ResNet-502023.05 | 87.3 | 83 | 78.6 | — | — | — | |
| MUTRBackbone=Video-Swin-T2023.05 | 87.2 | 83 | 78.7 | — | — | — | |
| BaselineBackbone=ResNet-502023.05 | 84.8 | 78.8 | 72.8 | — | — | — | |
| SSTBackbone=ResNet-502023.05 | 80.1 | 73.2 | 66.3 | — | — | — | |
| SOUPLEObject-guided refinement (OGL)=false, Zero-shot evaluation=true, Training epochs=502026.03 | 74 | — | — | — | — | 64.58 | |
| SOUPLEObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 71.47 | — | — | — | — | 62.89 | |
| ACL-SSLObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 69.03 | — | — | — | — | 59.76 | |
| LVSBackbone=ResNet-182023.05 | 51 | 44.5 | 37.9 | — | — | — | |
| MarginNCEObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 45.33 | — | — | — | — | 33.27 | |
| AudioCLIPObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 42.15 | — | — | — | — | 36.57 | |
| AlignmentObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 35.9 | — | — | — | — | 29.6 | |
| WAV2CLIPObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 35.35 | — | — | — | — | 28.7 | |
| SLAVCObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 34.6 | — | — | — | — | 28.1 | |
| FNACObject-guided refinement (OGL)=false, Zero-shot evaluation=true2026.03 | 31.4 | — | — | — | — | 27.15 | |
| AVMoEVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=54.4, Total Params(M)=501.2, Memory (GB)=9.52025.11 | — | — | — | 81.1 | 89.7 | — | |
| AVMoE2026.02 | — | — | — | 81.1 | 89.7 | — | |
| AVSVisual Encoder=PVT-V2, Audio Encoder=VGGish, Trainable Params(%)=58.7, Total Params(M)=174.5, Memory (GB)=4.82025.11 | — | — | — | 78.7 | 87.9 | — | |
| AVS2026.02 | — | — | — | 78.7 | 87.9 | — | |
| CAE-AV2026.02 | — | — | — | 81.9 | 89.9 | — | |
| CoPL2026.02 | — | — | — | 80.1 | — | — | |
| DG-SCTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=61.5, Total Params(M)=594.8, Memory (GB)=92025.11 | — | — | — | 80.9 | 89.2 | — | |
| DG-SCT2026.02 | — | — | — | 80.9 | 89.2 | — | |
| LAVisHVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=14, Total Params(M)=266.4, Memory (GB)=5.92025.11 | — | — | — | 80.1 | 88 | — | |
| LAVisHVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=47.6, Total Params(M)=389.72025.11 | — | — | — | 78 | 87 | — | |
| LAVisHentry=12026.02 | — | — | — | 80.1 | 88 | — | |
| LAVisHentry=22026.02 | — | — | — | 78 | 87 | — | |
| MettleVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=11.1, Total Params(M)=257.2, Memory (GB)=2.72025.11 | — | — | — | 79.9 | — | — | |
| MettleVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=21.6, Total Params(M)=291.7, Memory (GB)=2.92025.11 | — | — | — | 80.7 | — | — | |
| MoLTVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=8.3, Total Params(M)=313.1, Memory (GB)=3.62025.11 | — | — | — | 80.2 | 89.5 | — | |
| MoLTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=6.3, Total Params(M)=339.5, Memory (GB)=3.72025.11 | — | — | — | 80.8 | 89.9 | — |