# EEG2Mel: Reconstructing Sound from Brain Responses to Music

Source: [EEG2Mel: Reconstructing Sound from Brain Responses to Music](<https://www.sota2.com/research/paper/68fa6d211e209131cea37fd6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:26:36.450Z

Authors: Adolfo G. Ramirez-Aristizabal, Chris Kello

Published: 2022-07-28T01:06:51.000Z

## Abstract

Information retrieval from brain responses to auditory and visual stimuli has shown success through classification of song names and image classes presented to participants while recording EEG signals. Information retrieval in the form of reconstructing auditory stimuli has also shown some success, but here we improve on previous methods by reconstructing music stimuli well enough to be perceived and identified independently. Furthermore, deep learning models were trained on time-aligned music stimuli spectrum for each corresponding one-second window of EEG recording, which greatly reduces feature extraction steps needed when compared to prior studies. The NMED-Tempo and NMED-Hindi datasets of participants passively listening to full length songs were used to train and validate Convolutional Neural Network (CNN) regressors. The efficacy of raw voltage versus power spectrum inputs and linear versus mel spectrogram outputs were tested, and all inputs and outputs were converted into 2D images. The quality of reconstructed spectrograms was assessed by training classifiers which showed 81% accuracy for mel-spectrograms and 72% for linear spectrograms (10% chance accuracy). Lastly, reconstructions of auditory music stimuli were discriminated by listeners at an 85% success rate (50% chance) in a two-alternative match-to-sample task.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [EEG-to-Music Reconstruction on NMED-T and NMED-H](<https://www.sota2.com/research/sota/eeg-to-music-reconstruction-on-nmed-t-and-nmed-h.md>): EEG-to-Music Reconstruction; NMED-T and NMED-H; SSIM: 0.762.

- [EEG-Music Alignment on NMED-T and NMED-H](<https://www.sota2.com/research/sota/eeg-music-alignment-on-nmed-t-and-nmed-h.md>): EEG-Music Alignment; NMED-T and NMED-H; ID Accuracy (50-way): 25.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.13845>)

[PDF](<https://arxiv.org/pdf/2207.13845.pdf>)