# Multi-Loss Learning for Speech Emotion Recognition with Energy-Adaptive Mixup and Frame-Level Attention

Source: [Multi-Loss Learning for Speech Emotion Recognition with Energy-Adaptive Mixup and Frame-Level Attention](<https://www.sota2.com/research/paper/69326756d6b0dd3342934562>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-07T05:00:10.880Z

Authors: Cong Wang, Yizhong Geng, Yuhua Wen, Qifei Li, Yingming Gao, Ruimin Wang, Chunfeng Wang, Hao Li, Ya Li, Wei Chen

Published: 2025-12-04T08:04:45.000Z

## Abstract

Speech emotion recognition (SER) is an important technology in human-computer interaction. However, achieving high performance is challenging due to emotional complexity and scarce annotated data. To tackle these challenges, we propose a multi-loss learning (MLL) framework integrating an energy-adaptive mixup (EAM) method and a frame-level attention module (FLAM). The EAM method leverages SNR-based augmentation to generate diverse speech samples capturing subtle emotional variations. FLAM enhances frame-level feature extraction for multi-frame emotional cues. Our MLL strategy combines Kullback-Leibler divergence, focal, center, and supervised contrastive loss to optimize learning, address class imbalance, and improve feature separability. We evaluate our method on four widely used SER datasets: IEMOCAP, MSP-IMPROV, RAVDESS, and SAVEE. The results demonstrate our method achieves state-of-the-art performance, suggesting its effectiveness and robustness.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Speech Emotion Recognition on IEMOCAP Speaker-independent 5-fold cross-validation](<https://www.sota2.com/research/sota/speech-emotion-recognition-on-iemocap-speaker-independent-5-fold-cross-validation.md>): Speech Emotion Recognition; IEMOCAP Speaker-independent 5-fold cross-validation; WA: 78.47.

- [Speech Emotion Recognition on RAVDESS (6-fold subject-independent CV)](<https://www.sota2.com/research/sota/speech-emotion-recognition-on-ravdess-6-fold-subject-independent-cv.md>): Speech Emotion Recognition; RAVDESS (6-fold subject-independent cross-validation); Weighted Accuracy (WA): 93.4.

- [Speech Emotion Recognition on MSP-IMPROV (6-fold session-independent cross-validation)](<https://www.sota2.com/research/sota/speech-emotion-recognition-on-msp-improv-6-fold-session-independent-cross-validation.md>): Speech Emotion Recognition; MSP-IMPROV (6-fold session-independent cross-validation); Weighted Accuracy: 58.55.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2512.04551>)

[PDF](<https://arxiv.org/pdf/2512.04551.pdf>)