Multimodal Sentiment Analysis on CMU-MOSI Word Aligned (test)
43.3Accuracy (7-Class)TupleInfoNCE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TupleInfoNCEStrategy=Negative sample optimization2021.07 | 43.3 | 83.6 | 83.8 | — | — | |
| MISA2021.07 | 42.3 | 83.4 | 83.6 | — | — | |
| MMV FAC2021.07 | 41.5 | 83.5 | 83.4 | — | — | |
| InfoNCE2021.07 | 40.5 | 83.1 | 82.8 | — | — | |
| MulTalignment=Word Aligned2019.06 | 40 | 83 | 82.8 | 0.871 | 0.698 | |
| Train from scratch2021.07 | 40 | 83 | 82.8 | — | — | |
| Full modality (ours)Modality=Full modality2023.12 | 39.7 | 82.9 | 82.7 | 0.87 | 0.694 | |
| CMC2021.07 | 39.5 | 83.3 | 83 | — | — | |
| OursModality=Language and vision with reconstructed audio2023.12 | 39.1 | 82.3 | 82.2 | 0.858 | 0.691 | |
| Language and vision (ours)Modality=Language and vision2023.12 | 38.7 | 81.3 | 81.2 | 0.849 | 0.688 | |
| RMFNalignment=Word Aligned2019.06 | 38.3 | 78.4 | 78 | 0.922 | 0.681 | |
| RMFNModality=Full modality2023.12 | 38.3 | 78.4 | 78 | 0.922 | 0.681 | |
| Language only (ours)Modality=Language only2023.12 | 36.4 | 78.1 | 77.3 | 0.751 | 0.619 | |
| MFMalignment=Word Aligned2019.06 | 36.2 | 78.1 | 78.1 | 0.951 | 0.662 | |
| MFMModality=Full modality2023.12 | 36.2 | 78.1 | 78.1 | 0.951 | 0.662 | |
| MCTNalignment=Word Aligned2019.06 | 35.6 | 79.3 | 79.1 | 0.909 | 0.676 | |
| MCTN2021.11 | 35.6 | 79.3 | 79.1 | 0.909 | 0.676 | |
| MCTNModality=Full modality2023.12 | 35.6 | 79.3 | 79.1 | 0.909 | 0.676 | |
| LF-LSTMalignment=Word Aligned, fusion=Late Fusion2019.06 | 35.3 | 76.8 | 76.7 | 1.015 | 0.625 | |
| LF-LSTM2021.11 | 35.3 | 76.8 | 76.7 | 1.015 | 0.625 | |
| MoNIGmodality=pseudo2021.11 | 34.1 | 80.6 | 80.6 | 0.951 | 0.68 | |
| Vision only (ours)Modality=Vision only2023.12 | 34.1 | 76.6 | 76.4 | 0.721 | 0.421 | |
| EF-LSTMalignment=Word Aligned, fusion=Early Fusion2019.06 | 33.7 | 75.3 | 75.2 | 1.023 | 0.608 | |
| EF-LSTM2021.11 | 33.7 | 75.3 | 75.2 | 1.023 | 0.608 | |
| Evidential2021.11 | 33.4 | 77.6 | 77.6 | 0.974 | 0.655 | |
| RAVENalignment=Word Aligned2019.06 | 33.2 | 78 | 76.6 | 0.915 | 0.691 | |
| RAVEN2021.11 | 33.2 | 78 | 76.6 | 0.915 | 0.691 | |
| RAVENModality=Full modality2023.12 | 33.2 | 78 | 76.6 | 0.915 | 0.691 | |
| MoNIG2021.11 | 33 | 80.2 | 80.4 | 0.959 | 0.664 | |
| Gaussian2021.11 | 32.9 | 78.4 | 78.4 | 0.982 | 0.657 |