Musical Instrument Classification on NSynth
80.5AccuracyCLAP2023
Evaluation results
| Method | Links | |
|---|---|---|
| CLAP2023Audio enc. fine-tuning on AudioSet=true2025.03 | 80.5 | |
| AaPE#Params=87M, Protocol=Linear Probing2025.12 | 80.5 | |
| ATST-F2C# Param=86M, Pre-training data=AS, knowledge distillation=true2023.06 | 79.8 | |
| ATST-Frame# Param=86M, Pre-training data=AS2023.06 | 79.2 | |
| ATST-C2F# Param=86M, Pre-training data=AS, knowledge distillation=true2023.06 | 79.2 | |
| Chu et al., 20232024.02 | 78.8 | |
| ATST-Clip# Param=86M, Pre-training data=AS2023.06 | 78.6 | |
| DeLoRes-MNumber of Parameters=5.3M, Linear Evaluation=true2024.04 | 78.2 | |
| M2D-CLAP stage1 2024Audio enc. fine-tuning on AudioSet=false2025.03 | 78 | |
| CLAP-UEvaluation Protocol=kNN2026.02 | 78 | |
| AST-Fusion#5#12Evaluation protocol=Linear evaluation, Variant=#5#122022.10 | 77.6 | |
| AST-Fusion#5#12Number of Parameters=86M, Linear Evaluation=true2024.04 | 77.6 | |
| CLAP-UEvaluation Protocol=MLP2026.02 | 77.3 | |
| Audio Flamingo2024.02 | 77.1 | |
| M2DEvaluation protocol=Linear evaluation, Masking ratio=0.72022.10 | 76.9 | |
| M2Dratio=0.7, Training data=AS, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 76.9 | |
| M2DNumber of Parameters=86M, Masking ratio=0.7, Target Feeding=all patches, Linear Evaluation=true2024.04 | 76.8 | |
| MATPAC++Role=Teacher, Size=86M2026.04 | 76.8 | |
| M2D-CLAP2025Audio enc. fine-tuning on AudioSet=true2025.03 | 76.7 | |
| M2D-CLAP stage1 2025Audio enc. fine-tuning on AudioSet=false2025.03 | 76.5 | |
| ATST-ClipAudio enc. fine-tuning on AudioSet=false2025.03 | 76.2 | |
| ATST BaseNumber of Parameters=86M, Linear Evaluation=true2024.04 | 76.2 | |
| ATST-ClipTraining data=AS, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 76.2 | |
| M2DRole=Teacher, Size=86M2026.04 | 76.2 | |
| M2DNumber of Parameters=86M, Masking ratio=0.6, Target Feeding=masked patches, Linear Evaluation=true2024.04 | 76.1 | |
| ATST-FrameAudio enc. fine-tuning on AudioSet=false2025.03 | 75.9 | |
| BEATsiter3Audio enc. fine-tuning on AudioSet=false2025.03 | 75.9 | |
| MSM-MAEEvaluation protocol=Linear evaluation2022.10 | 75.9 | |
| BEATSiter3Number of Parameters=90M, Linear Evaluation=true2024.04 | 75.9 | |
| ATST-FrameTraining data=AS, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 75.9 | |
| BEATs_iter3#Params=90M, Protocol=Linear Probing2025.12 | 75.9 | |
| ATST-Frame#Params=86M, Protocol=Linear Probing2025.12 | 75.9 | |
| BEATsLabel=-, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 75.9 | |
| M2DAudio enc. fine-tuning on AudioSet=false2025.03 | 75.7 | |
| M2D# Params=86M, Masking ratio=0.7, Evaluation protocol=Linear evaluation2024.04 | 75.7 | |
| M2DNumber of Parameters=86M, Masking ratio=0.7, Target Feeding=masked patches, Linear Evaluation=true2024.04 | 75.7 | |
| M2D/0.7#Params=86M, Protocol=Linear Probing2025.12 | 75.7 | |
| CEDAudio enc. fine-tuning on AudioSet=false2025.03 | 75.6 | |
| M2DASAudio enc. fine-tuning on AudioSet=true2025.03 | 75.6 | |
| ATST BaseEvaluation protocol=Linear evaluation, Variant=Base2022.10 | 75.6 | |
| ATST-BaseEvaluation protocol=Linear evaluation2022.04 | 75.6 | |
| M2DEvaluation protocol=Linear evaluation, Masking ratio=0.62022.10 | 75.3 | |
| ATST-SmallEvaluation protocol=Linear evaluation2022.04 | 75.3 | |
| M2D-AS# Params=86M, Masking ratio=0.7, Evaluation protocol=Linear evaluation2024.04 | 75.2 | |
| M2DNumber of Parameters=86M, Masking ratio=0.6, Target Feeding=all patches, Linear Evaluation=true2024.04 | 75.2 | |
| DeLoRes-MEvaluation protocol=Linear evaluation2022.10 | 75 | |
| S-SONDO (MobileNetV3)Teacher=MATPAC++, Size=2.9M2026.04 | 74.9 | |
| MATPACAudio enc. fine-tuning on AudioSet=false2025.03 | 74.6 | |
| MSM-MAENumber of Parameters=86M, Masking ratio=0.75, Linear Evaluation=true2024.04 | 74.6 | |
| MATPACTraining data=AS, nr,epoch=20, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 74.6 | |
| WavCapsAudio enc. fine-tuning on AudioSet=true2025.03 | 74.4 | |
| MATPACTraining data=AS, nr,epoch=10, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 74.3 | |
| MATPAC/n10#Params=86M, Protocol=Linear Probing2025.12 | 74.3 | |
| BYOL-AEvaluation protocol=Linear evaluation2022.04 | 74.1 | |
| MATPAC w/o ClassificationTraining data=AS, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 73.7 | |
| HTS-ATAudio enc. fine-tuning on AudioSet=true2025.03 | 73.3 | |
| HTS-AT# Params=31M, Evaluation protocol=Linear evaluation2024.04 | 73.3 | |
| HTS-ATNumber of Parameters=31M, Linear Evaluation=true2024.04 | 73.3 | |
| AST# Params=86M, Evaluation protocol=Linear evaluation2024.04 | 73.2 | |
| ASTNumber of Parameters=86M, Linear Evaluation=true2024.04 | 73.2 | |
| BEATsiter3+Audio enc. fine-tuning on AudioSet=true2025.03 | 73.1 | |
| BYOL-AEvaluation protocol=Linear evaluation2022.10 | 73.1 | |
| BEATSiter3+# Params=90M, Evaluation protocol=Linear evaluation2024.04 | 73.1 | |
| BYOL-ANumber of Parameters=5.3M, Linear Evaluation=true2024.04 | 73.1 | |
| BEATSiter3+Number of Parameters=90M, Linear Evaluation=true2024.04 | 73.1 | |
| S-SONDO (ERes2Net)Teacher=MATPAC++, Size=1.4M2026.04 | 73 | |
| ASTAudio enc. fine-tuning on AudioSet=true2025.03 | 72.9 | |
| PaSSTTraining data=AS, Learning type=Supervised, Evaluation protocol=Linear evaluation2025.02 | 72.9 | |
| MERTLabel=-, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 72.6 | |
| BEATS iter3Training data=AS, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 72.4 | |
| S-SONDO (ERes2Net)Teacher=M2D, Size=1.4M2026.04 | 72.4 | |
| LAION-CLAPAudio enc. fine-tuning on AudioSet=true2025.03 | 72.2 | |
| STFT-MelFilterbank=STFT-Mel*, Compression=L-M-TBN*2022.07 | 72.1 | |
| S-SONDO (DyMN)Teacher=MATPAC++, Size=8.7M2026.04 | 72.1 | |
| EfficientLEAFFilterbank=Gabor 8G-opt, Compression=L-M-TBN2022.07 | 71.7 | |
| BEATs iter3+Training data=AS, Learning type=Supervised, Evaluation protocol=Linear evaluation2025.02 | 71.6 | |
| MAE-ASTTraining data=AS+LS, Learning type=SSL, Evaluation protocol=Linear evaluation2025.02 | 71.2 | |
| UniWhisperEvaluation Protocol=MLP2026.02 | 70.7 | |
| EfficientLEAFFilterbank=Gabor 4G, Compression=L-M-TBN2022.07 | 70.4 | |
| UniWhisperEvaluation Protocol=kNN2026.02 | 70 | |
| S-SONDO (MobileNetV3)Teacher=M2D, Size=2.9M2026.04 | 69.5 | |
| LEAFFilterbank=Gabor, Compression=PCEN2022.07 | 69.2 | |
| HTS-ATTraining data=AS, Learning type=Supervised, Evaluation protocol=Linear evaluation2025.02 | 68.6 | |
| EfficientLEAFFilterbank=Gabor 4G, Compression=PCEN2022.07 | 68.3 | |
| CLAP2022Audio enc. fine-tuning on AudioSet=true2025.03 | 68.2 | |
| MobileNetV3Teacher=None (Supervised), Size=2.9M2026.04 | 68 | |
| DyMNTeacher=None (Supervised), Size=8.7M2026.04 | 67.5 | |
| S-SONDO (DyMN)Teacher=M2D, Size=8.7M2026.04 | 67.3 | |
| ERes2NetTeacher=None (Supervised), Size=1.4M2026.04 | 67.3 | |
| MTC (AudioSet)Label=Tag, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 67.19 | |
| PANNs CNN14Audio enc. fine-tuning on AudioSet=true2025.03 | 65.9 | |
| BEATsEvaluation Protocol=MLP2026.02 | 64.8 | |
| MTC (Ours-UTS)Label=Tag*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 63.62 | |
| COLAEvaluation protocol=Linear evaluation2022.04 | 63.4 | |
| BEATsEvaluation Protocol=kNN2026.02 | 61.6 | |
| Contrastive-scratch (Ours)Label=Caption*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 61.4 | |
| Contrastive-scratchLabel=Caption, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 60.91 | |
| Multi-Task (Ours)Label=Tag*, Caption*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 59.94 | |
| F3-TokenizerProbing=Frozen representation2026.06 | 58.12 | |
| PAR (Ours-UTS)Label=Tag Sequence*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 57.91 |