Emotion Recognition on CREMA-D
84.41Overall AccuracyCrab+
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Crab+Evaluation Setting=First split2026.03 | 84.41 | — | — | — | |
| CALEvaluation Setting=First split2026.03 | 79.3 | — | — | — | |
| MARSAudio=Presence, Visual=Presence2026.06 | 76.1 | — | — | — | |
| Audio-visual fusion2026.05 | 74 | — | — | — | |
| Audio-video (Gating Paradigm)Inputs=Audio-video (Gating Paradigm)2026.05 | 74 | — | — | — | |
| Concatenation of Audio (LSTM) and Video (LSTM)Inputs=Concatenation of Audio (LSTM) and Video (LSTM)2026.05 | 72.9 | — | — | — | |
| InfoRegBackbone=ResNet-18, Loss type=multimodal loss only2025.03 | 71.9 | 60.03 | 49.65 | — | |
| MMParetoEvaluation Setting=First split2026.03 | 70.19 | — | — | — | |
| DMRNetAudio=Presence, Visual=Presence2026.06 | 70.1 | — | — | — | |
| AGMBackbone=ResNet-18, Loss type=multimodal loss only2025.03 | 69.71 | 59.32 | 43.72 | — | |
| Audio-videoInputs=Audio-video2026.05 | 69 | — | — | — | |
| OGMBackbone=ResNet-18, Loss type=multimodal loss only2025.03 | 68.7 | 56.84 | 39.52 | — | |
| MMANetAudio=Presence, Visual=Presence2026.06 | 67.87 | — | — | — | |
| GreedyBackbone=ResNet-18, Loss type=multimodal loss only2025.03 | 67.82 | 59.17 | 40.17 | — | |
| Contrastive-scratch (Ours)Label=Caption*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 67.29 | — | — | — | |
| MTC (AudioSet)Label=Tag, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 67.14 | — | — | — | |
| PMRBackbone=ResNet-18, Loss type=multimodal loss only2025.03 | 66.92 | 57.83 | 38.91 | — | |
| Video Only (LSTM)Inputs=Video Only (LSTM)2026.05 | 66.8 | — | — | — | |
| ShaSpecAudio=Presence, Visual=Presence2026.06 | 66.8 | — | — | — | |
| Joint trainingBackbone=ResNet-18, Loss type=multimodal loss only2025.03 | 66.61 | 58.99 | 35.14 | — | |
| MTC (Ours-UTS)Label=Tag*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 66.01 | — | — | — | |
| MARSAverage=true2026.06 | 65.52 | — | — | — | |
| Multi-Task (Ours)Label=Tag*, Caption*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 65.31 | — | — | — | |
| Video Only (Gating Paradigm)Inputs=Video Only (Gating Paradigm)2026.05 | 65 | — | — | — | |
| Captioning-scratch (Ours)Label=Caption*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 64.74 | — | — | — | |
| OGM-GEEvaluation Setting=First split2026.03 | 64.34 | — | — | — | |
| Contrastive-scratchLabel=Caption, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 63.74 | — | — | — | |
| Video OnlyInputs=Video Only2026.05 | 63.1 | — | — | — | |
| PAR (Ours-UTS)Label=Tag Sequence*, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 62.47 | — | — | — | |
| DMRNetAverage=true2026.06 | 61.35 | — | — | — | |
| MARSAudio=Presence, Visual=Absence2026.06 | 61.04 | — | — | — | |
| Captioning-scratchLabel=Caption, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 60.91 | — | — | — | |
| ShaSpecAudio=Presence, Visual=Absence2026.06 | 59.86 | — | — | — | |
| MARSAudio=Absence, Visual=Presence2026.06 | 59.43 | — | — | — | |
| MMANetAudio=Presence, Visual=Absence2026.06 | 58.89 | — | — | — | |
| DMRNetAudio=Presence, Visual=Absence2026.06 | 58.87 | — | — | — | |
| ShaSpecAverage=true2026.06 | 57.94 | — | — | — | |
| MMANetAverage=true2026.06 | 57.66 | — | — | — | |
| Audio Only (Gating Paradigm)Inputs=Audio Only (Gating Paradigm)2026.05 | 57 | — | — | — | |
| Audio OnlyInputs=Audio Only2026.05 | 56.4 | — | — | — | |
| Wav2vec 2.0Label=-, Evaluation Protocol=linear probing, Pooling Strategy=mean-pooling2026.03 | 56.1 | — | — | — | |
| DMRNetAudio=Absence, Visual=Presence2026.06 | 55.1 | — | — | — | |
| Audio Only (LSTM)Inputs=Audio Only (LSTM)2026.05 | 50.2 | — | — | — | |
| MMANetAudio=Absence, Visual=Presence2026.06 | 47.31 | — | — | — | |
| ShaSpecAudio=Absence, Visual=Presence2026.06 | 47.17 | — | — | — | |
| PALM + ASPL*Shots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 43.41 | — | — | — | |
| CoCoOp + ASPLShots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 40.45 | — | — | — | |
| CoCoOpShots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 39.87 | — | — | — | |
| CoOpShots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 39.44 | — | — | — | |
| PALM + ASPLShots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 38.08 | — | — | — | |
| CoCoOp + ASPL*Shots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 38.06 | — | — | — | |
| CoOp + ASPL*Shots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 37.75 | — | — | — | |
| PALMShots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 36.15 | — | — | — | |
| CoOp + ASPLShots=16, Audio Encoder=CLAP-HTSAT, Text Encoder=CLIP (PENGI)2026.06 | 35.12 | — | — | — | |
| ARLMethod category=Gradient-Based2025.12 | — | — | — | 63.18 | |
| AudioMethod category=Unimodal2025.12 | — | — | — | 60.81 | |
| AUGMethod category=Encoder-Based2025.12 | — | — | — | 61.05 | |
| BMMLMethod category=Gradient-Based2025.12 | — | — | — | 65.35 | |
| Coupled MambaMethod category=Fusion-Based2025.12 | — | — | — | 67.92 | |
| DGLMethod category=Gradient-Based2025.12 | — | — | — | 67.17 | |
| G-Blend OfflineMethod category=Gradient-Based2025.12 | — | — | — | 62.41 | |
| G-Blend OnlineMethod category=Gradient-Based2025.12 | — | — | — | 55.66 | |
| IMDerMethod category=Missing Data-Based2025.12 | — | — | — | 68.4 | |
| LMFMethod category=Fusion-Based2025.12 | — | — | — | 63.63 | |
| MBTMethod category=Fusion-Based2025.12 | — | — | — | 65.25 | |
| MMPMethod category=Missing Data-Based2025.12 | — | — | — | 65.52 | |
| MMParetoMethod category=Gradient-Based2025.12 | — | — | — | 43.98 | |
| MulTMethod category=Head-Based2025.12 | — | — | — | 61.02 | |
| OGMMethod category=Gradient-Based2025.12 | — | — | — | 61.85 | |
| OGMGEMethod category=Gradient-Based2025.12 | — | — | — | 62.81 | |
| OMIBMethod category=Fusion-Based2025.12 | — | — | — | 66.9 | |
| PDFMethod category=Fusion-Based2025.12 | — | — | — | 59.76 | |
| RegBNMethod category=Encoder-Based2025.12 | — | — | — | 63.4 | |
| ShaSpecMethod category=Missing Data-Based2025.12 | — | — | — | 60.51 | |
| SimBaMMCLSMethod category=Missing Data-Based2025.12 | — | — | — | 63.8 | |
| SimBaMMCLSMethod category=Proposed2025.12 | — | — | — | 67.23 | |
| VisionMethod category=Unimodal2025.12 | — | — | — | 31.92 |