# Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognition Source: [Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognition]() Collection index: [llms.txt]() Data updated: 2025-10-25T23:52:06.013Z Authors: Yi Zhang, Mingyuan Chen, Jundong Shen, Chongjun Wang Published: 2022-01-15T12:02:28.000Z ## Abstract Multi-modal Multi-label Emotion Recognition (MMER) aims to identify various human emotions from heterogeneous visual, audio and text modalities. Previous methods mainly focus on projecting multiple modalities into a common latent space and learning an identical representation for all labels, which neglects the diversity of each modality and fails to capture richer semantic information for each label from different perspectives. Besides, associated relationships of modalities and labels have not been fully exploited. In this paper, we propose versaTile multi-modAl learning for multI-labeL emOtion Recognition (TAILOR), aiming to refine multi-modal representations and enhance discriminative capacity of each label. Specifically, we design an adversarial multi-modal refinement module to sufficiently explore the commonality among different modalities and strengthen the diversity of each modality. To further exploit label-modal dependence, we devise a BERT-like cross-modal encoder to gradually fuse private and common modality representations in a granularity descent way, as well as a label-guided decoder to adaptively generate a tailored representation for each label with the guidance of label semantics. In addition, we conduct experiments on the benchmark MMER dataset CMU-MOSEI in both aligned and unaligned settings, which demonstrate the superiority of TAILOR over the state-of-the-arts. Code is available at https://github.com/kniter1/TAILOR. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Emotional Recognition on EMER 7-class (test)](): Emotional Recognition; EMER 7-class (test); Weighted Accuracy (WAR): 33.39. - [Facial Expression Recognition on EMER 7-class (test)](): Facial Expression Recognition; EMER 7-class (test); WAR: 48.9. - [Facial Expression Recognition on EMER 3-class (test)](): Facial Expression Recognition; EMER 3-class (test); WAR: 63.91. - [Emotion Recognition on EMER 3-class (test)](): Emotional Recognition; EMER 3-class (test); WAR: 52.98. - [Valence and Arousal Regression on EMER (test)](): Emotion Recognition (ER) Valence and Arousal Regression; EMER (test); Arousal MAE: 0.246. ## Original sources and implementation [Original paper]() [PDF]()