Sound Classification on AudioSet (evaluation)
47.1mAPHTS-AT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| HTS-ATPretrain=true, Parameters=31M, Variant=HCP, Description=hierarchical structure, token-semantic module, and pretrained vision model2022.02 | 47.1 | — | — | 0.487 | |
| Attention Audiovisual FusionModality=AudioVisual2020.05 | 46.16 | — | 97.51 | — | |
| ASTPretrain=true, Parameters=87M, Variant=AST-P2022.02 | 45.9 | — | — | 0.475 | |
| ERANN-1-6e2e=No, Pretrained=none, #Parameters (x10^6)=54.5, Inference time (msec)=14, Hardware=V-1002022.04 | 45.6 | — | — | — | |
| HTS-ATPretrain=false, Parameters=31M, Variant=HC, Description=hierarchical structure and token-semantic module2022.02 | 45.3 | — | — | — | |
| HTS-ATe2e=No, Pretrained=none, #Parameters (x10^6)=312022.04 | 45.3 | — | — | — | |
| PSLAPretrain=true, Parameters=13.6M, Variant=PSLA-P2022.02 | 44.4 | — | — | 0.474 | |
| HTS-ATPretrain=false, Parameters=28.8M, Variant=H, Description=only hierarchical structure2022.02 | 44 | — | — | — | |
| PANNPretrain=false, Parameters=81M2022.02 | 43.4 | — | — | — | |
| EAT-Me2e=Yes, Pretrained=none, #Parameters (x10^6)=25.5, Inference time (msec)=14.62022.04 | 42.6 | — | — | — | |
| G-BlendModality=AudioVisual2020.05 | 41.8 | — | 97.5 | — | |
| EAT-Se2e=Yes, Pretrained=none, #Parameters (x10^6)=5.3, Inference time (msec)=8.42022.04 | 40.5 | — | — | — | |
| DeepResPretrain=false, Parameters=26M2022.02 | 39.2 | — | — | — | |
| Audio2020.05 | 38.35 | — | 97.12 | — | |
| AudioBackbone=ResNet-502020.05 | 38 | — | 97.1 | — | |
| ASTPretrain=false, Parameters=87M2022.02 | 36.6 | — | — | — | |
| ASTe2e=No, Pretrained=none, #Parameters (x10^6)=88.1, Inference time (msec)=63.52022.04 | 36.6 | — | — | — | |
| data2vecPretrain data=AudioSet (AS), Fine-tuning subset=20K2022.02 | 34.5 | — | — | — | |
| AemNet WM1.0e2e=Yes, Pretrained=none, #Parameters (x10^6)=52022.04 | 33.16 | — | — | — | |
| MaskSpecPretrain data=AudioSet (AS), Fine-tuning subset=20K2022.02 | 32.3 | — | — | — | |
| BaselinePretrain=false, Parameters=2.6M2022.02 | 31.4 | — | — | — | |
| SSASTPretrain data=AudioSet (AS), Librispeech (LS), Fine-tuning subset=20K2022.02 | 31 | — | — | — | |
| MAE-ASTPretrain data=AudioSet (AS), Librispeech (LS), Fine-tuning subset=20K2022.02 | 30.6 | — | — | — | |
| Explicit Label Triplet (topline)Classifier Architecture=2 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 28.9 | 100 | — | — | |
| Explicit Label Triplet (topline)Classifier Architecture=1 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 28.8 | 100 | — | — | |
| Joint Unsupervised TripletClassifier Architecture=2 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 25.9 | 84 | — | — | |
| Visual2020.05 | 25.73 | — | 91.3 | — | |
| Joint Unsupervised TripletClassifier Architecture=1 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 24.4 | 80 | — | — | |
| Temporal Proximity (Δt=10s)Classifier Architecture=2 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 24.1 | 74 | — | — | |
| Temporal Proximity (Δt=10s)Classifier Architecture=1 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 22.6 | 72 | — | — | |
| VisualBackbone=R(2+1)D-1012020.05 | 18.8 | — | 91.8 | — | |
| T/F Translation (S = 10)Classifier Architecture=2 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 12.5 | 12 | — | — | |
| Mixed Example (α=0.25)Classifier Architecture=2 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 12.2 | 11 | — | — | |
| Gaussian Noise (σ=0.5)Classifier Architecture=2 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 11.4 | 6 | — | — | |
| T/F Translation (S = 10)Classifier Architecture=1 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 10.8 | 19 | — | — | |
| Mixed Example (α=0.25)Classifier Architecture=1 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 10.3 | 17 | — | — | |
| Log Mel Spectrogram (baseline)Classifier Architecture=2 layer, 512 units, Embedding Dimension=61442017.11 | 10.2 | 0 | — | — | |
| Gaussian Noise (σ=0.5)Classifier Architecture=1 layer, 512 units, Backbone=ResNet-50, Embedding Dimension=1282017.11 | 9.6 | 14 | — | — | |
| Log Mel Spectrogram (baseline)Classifier Architecture=1 layer, 512 units, Embedding Dimension=61442017.11 | 6.5 | 0 | — | — |