List your company
ResearchFormerly WizWand

Audio-Visual Event Localization on AVE

85AccuracyTB-AVA (Ours)
63.88869.36974.8580.331Dec 15, 2022Jul 10, 2023Feb 2, 2024Aug 28, 2024Mar 23, 2025Oct 16, 2025May 12, 2026
Updated 5mo ago

Evaluation results

MethodLinks
TB-AVA (Ours)Visual Encoder=SigLIP2-L/16, Audio Encoder=BEATs, Trainable Params (%)=16.7, Total Params (M)=488.12026.0585—
Crab+2026.0383.58—
MoLT2026.0383.5—
MoLTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params (%)=6.2, Total Params (M)=290.32026.0583.5—
MettleVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params (%)=23.0, Total Params (M)=338.02026.0583.3—
TB-AVA without GSMVisual Encoder=SigLIP2-L/16, Audio Encoder=BEATs, Trainable Params (%)=16.4, Total Params (M)=486.52026.0583.1—
DG-SCTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params (%)=43.6, Total Params (M)=461.32026.0581.2—
LAVISHVisual Encoder=Swin-V2-L (frozen), Audio Encoder=(shared) (frozen), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=None, Trainable Params (M)=10.1, Total Params (M)=238.82022.1281.1—
LAVisHVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params (%)=4.1, Total Params (M)=238.82026.0580.8—
UWAVBackbone=CLIP, R(2+1)D-18, and CLAP2025.0580.6—
CMBSImplementation=Improved implementation, Visual Encoder=Swin-V2-L (frozen), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=14.1, Total Params (M)=315.22022.1280.4—
VALORFeatures=CLAP, CLIP, R(2+1)D2023.0580.4—
VALORBackbone=CLIP, R(2+1)D-18, and CLAP2025.0580.4—
Crab2025.0380.15—
AVMoEVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params (%)=34.9, Total Params (M)=404.02026.0580—
DPNetVisual Encoder=VGG-19, Audio Encoder=VGGish, Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet2022.1279.7—
CMBSVisual Encoder=ResNet-152 (frozen), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=14.4, Total Params (M)=216.72022.1279.7—
CMBSVisual Encoder=ResNet-151, Audio Encoder=VGG-like, Trainable Params (%)=6.6, Total Params (M)=216.72026.0579.7—
CMBSImplementation=Improved implementation, Visual Encoder=Swin-V2-L (fully fine-tuned), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=243.1, Total Params (M)=315.22022.1279.6—
CoPLVisual Encoder=ViT-L/16 (shared), Audio Encoder=ViT-L/16 (shared), Trainable Params (%)=1.5, Total Params (M)=332.82026.0579.2—
LAVISHVisual Encoder=Swin-V2-B (frozen), Audio Encoder=(shared) (frozen), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=None, Trainable Params (M)=5, Total Params (M)=114.22022.1278.8—
CMRANVisual Encoder=ResNet-152 (frozen), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=15.9, Total Params (M)=148.22022.1278.3—
CMRANVisual Encoder=ResNet-151, Audio Encoder=VGG-like, Trainable Params (%)=10.7, Total Params (M)=148.22026.0578.3—
LAVISHVisual Encoder=ViT-L-16 (frozen), Audio Encoder=(shared) (frozen), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=None, Trainable Params (M)=14.5, Total Params (M)=340.12022.1278.1—
LAVisHVisual Encoder=ViT-L/16 (shared), Audio Encoder=ViT-L/16 (shared), Trainable Params (%)=4.3, Total Params (M)=340.12026.0578.1—
PSPVisual Encoder=VGG-19 (fully fine-tuned), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=1.7, Total Params (M)=217.42022.1277.8—
MM-PyramidVisual Encoder=ResNet-152 (frozen), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=44, Total Params (M)=176.32022.1277.8—
MBTVisual Encoder=ViT-B-16 (frozen), Audio Encoder=AST (frozen), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=172, Total Params (M)=1722022.1277.8—
MM-Pyramid2026.0377.8—
MBTVisual Encoder=ViT-B/16, Audio Encoder=AST, Trainable Params (%)=100, Total Params (M)=172.02026.0577.8—
AVTVisual Encoder=VGG-19 (fully fine-tuned), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=15.8, Total Params (M)=231.52022.1276.8—
CMBSFeatures=VGG-like, Res-1512023.0576—
AVT2026.0375.8—
AVSDNVisual Encoder=ResNet-152 (frozen), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=8, Total Params (M)=140.32022.1275.4—
LAVISHVisual Encoder=ViT-B-16 (frozen), Audio Encoder=(shared) (frozen), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=None, Trainable Params (M)=4.7, Total Params (M)=107.22022.1275.3—
CMRANFeatures=VGG-like, Res-1512023.0575.3—
HANFeatures=CLAP, CLIP, R(2+1)D2023.0575.3—
HANBackbone=CLIP, R(2+1)D-18, and CLAP2025.0575.3—
CMBSFeatures=VGG-like, VGG-192023.0574.2—
AVSDNFeatures=VGG-like, Res-1512023.0574.2—
AVELVisual Encoder=ResNet-152 (frozen), Audio Encoder=VGGish (fully fine-tuned), Visual Pretrain Dataset=ImageNet, Audio Pretrain Dataset=AudioSet, Trainable Params (M)=3.7, Total Params (M)=1362022.1274—
PSPFeatures=VGG-like, VGG-192023.0573.5—
CMRANFeatures=VGG-like, VGG-192023.0572.9—
AVELVisual Encoder=VGG-19, Audio Encoder=VGG-like, Trainable Params (%)=2.7, Total Params (M)=136.02026.0572.7—
AVELFeatures=VGG-like, Res-1512023.0571.6—
CMANFeatures=VGG-like, VGG-192023.0570.4—
AVTFeatures=VGG-like, VGG-192023.0570.2—
AVINFeatures=VGG-like, VGG-192023.0569.4—
AVRBFeatures=VGG-like, VGG-192023.0568.9—
AVSDNFeatures=VGG-like, VGG-192023.0567.3—
AVELFeatures=VGG-like, VGG-192023.0566.7—
DG-SCTProtocol=Zero-shot2025.0364.7—
Drop-DTWLearning Paradigm=Unsupervised2021.08—41.1
OTAMLearning Paradigm=Unsupervised2021.08—37.5
SmoothDTWLearning Paradigm=Unsupervised2021.08—39.8
SupervisedLearning Paradigm=Supervised2021.08—44.8