# Multilevel Transformer For Multimodal Emotion Recognition Source: [Multilevel Transformer For Multimodal Emotion Recognition]() Collection index: [llms.txt]() Data updated: 2025-10-26T01:02:50.787Z Authors: Junyi He, Meimei Wu, Meng Li, Xiaobo Zhu, Feng Ye Published: 2022-10-26T10:31:24.000Z ## Abstract Multimodal emotion recognition has attracted much attention recently. Fusing multiple modalities effectively with limited labeled data is a challenging task. Considering the success of pre-trained model and fine-grained nature of emotion expression, it is reasonable to take these two aspects into consideration. Unlike previous methods that mainly focus on one aspect, we introduce a novel multi-granularity framework, which combines fine-grained representation with pre-trained utterance-level representation. Inspired by Transformer TTS, we propose a multilevel transformer model to perform fine-grained multimodal emotion recognition. Specifically, we explore different methods to incorporate phoneme-level embedding with word-level embedding. To perform multi-granularity learning, we simply combine multilevel transformer model with Albert. Extensive experimental results show that both our multilevel transformer model and multi-granularity model outperform previous state-of-the-art approaches on IEMOCAP dataset with text transcripts and speech signal. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Speech Emotion Recognition on IEMOCAP Speaker-independent 5-fold cross-validation](): Speech Emotion Recognition; IEMOCAP Speaker-independent 5-fold cross-validation; WA: 74.5. ## Original sources and implementation [Original paper]() [PDF]()