Speech Emotion Recognition on IEMOCAP (five-fold/ten-fold cross-validation)
77.64WAemotion2vec
Evaluation Results
| Method | Links | |
|---|---|---|
| emotion2vecModel Scale=base size, Pre-training Corpus=LS-960 + Emo-262, #Upstream Params=93.79M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=ten-fold leave-one-speaker-out, Same validation and test fold=true2023.12 | 77.64 | |
| emotion2vecModel Scale=base size, Pre-training Corpus=LS-960 + Emo-262, #Upstream Params=93.79M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out, Same validation and test fold=true2023.12 | 74.48 | |
| emotion2vecModel Scale=base size, Pre-training Corpus=LS-960 + Emo-262, #Upstream Params=93.79M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=ten-fold leave-one-speaker-out, Same validation and test fold=false2023.12 | 72.94 | |
| DSTModel Scale=Supervised Model, Upstream=WavLM-large, #Upstream Params=316.62M, Downstream=Transformer(DST), #Downstream Params=22.78M2023.12 | 71.8 | |
| emotion2vecModel Scale=base size, Pre-training Corpus=LS-960 + Emo-262, #Upstream Params=93.79M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out, Same validation and test fold=false2023.12 | 71.79 | |
| Vesper-12Model Scale=base size, Pre-training Corpus=Mix-94k + LSED-206, #Upstream Params=164.29M, Downstream=Linear, #Downstream Params=0.26M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 70.7 | |
| WavLMModel Scale=large size, Pre-training Corpus=Mix-94k, #Upstream Params=316.62M, Downstream=Linear, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 70.03 | |
| MSTRModel Scale=Supervised Model, Upstream=HuBERT-large, #Upstream Params=316.61M, Downstream=Transformer(MSTR), #Downstream Params=27.00M2023.12 | 70.03 | |
| data2vec 2.0Model Scale=base size, Pre-training Corpus=LS-960, #Upstream Params=93.78M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 68.58 | |
| Vesper-4Model Scale=base size, Pre-training Corpus=Mix-94k + LSED-206, Upstream=Proposed, #Upstream Params=63.52M, Downstream=Linear, #Downstream Params=0.26M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 68.4 | |
| HuBERT#Param. (M)=95, Data (hrs)=960, Evaluation Protocol=Fine-tuning2026.04 | 68.3 | |
| TIM-NetModel Scale=Supervised Model, Upstream=MFCC, Downstream=CNN(TIM-Net), #Downstream Params=0.40M2023.12 | 68.29 | |
| WavLM+Model Scale=base size, Pre-training Corpus=Mix-94k, #Upstream Params=94.70M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 67.98 | |
| EnCodecMAE#Param. (M)=86.6, Data (hrs)=11,000, Evaluation Protocol=Fine-tuning2026.04 | 67.8 | |
| ULTRAS#Param. (M)=89, Data (hrs)=2,000, Evaluation Protocol=Fine-tuning2026.04 | 67.78 | |
| HUBERTModel Scale=large size, Pre-training Corpus=LL-60k, Upstream=Proposed, #Upstream Params=316.61M, Downstream=Linear, #Downstream Params=0.26M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 67.62 | |
| BEATs#Param. (M)=91, Data (hrs)=5,800, Evaluation Protocol=Fine-tuning2026.04 | 67.61 | |
| data2vecModel Scale=base size, Pre-training Corpus=LS-960, #Upstream Params=93.75M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 67.38 | |
| WavLMModel Scale=base size, Pre-training Corpus=LS-960, #Upstream Params=94.70M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 65.94 | |
| wav2vec 2.0Model Scale=large size, Pre-training Corpus=LL-60k, #Upstream Params=317.38M, Downstream=Linear, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 65.64 | |
| HUBERTModel Scale=base size, Pre-training Corpus=LS-960, #Upstream Params=94.68M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 64.92 | |
| wav2vec 2.0Model Scale=base size, Pre-training Corpus=LS-960, #Upstream Params=95.04M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 63.43 | |
| wav2vecModel Scale=small size, Pre-training Corpus=LS-960, Upstream=Proposed, #Upstream Params=32.54M, Downstream=Linear, #Downstream Params=0.13M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 59.79 | |
| vq-wav2vecModel Scale=small size, Pre-training Corpus=LS-960, Upstream=Proposed, #Upstream Params=34.15M, Downstream=Linear, #Downstream Params=0.20M, Evaluation Protocol=five-fold leave-one-session-out2023.12 | 58.24 | |
| SSAST#Param. (M)=89, Data (hrs)=5,800, Evaluation Protocol=Fine-tuning2026.04 | 55.21 |