# Measuring Robustness of Speech Recognition from MEG Signals Under Distribution Shift

Source: [Measuring Robustness of Speech Recognition from MEG Signals Under Distribution Shift](<https://www.sota2.com/research/paper/69d48fd77d0172140ec74679>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-08T05:00:12.103Z

Authors: Sheng-You Chien, Bo-Yi Mao, Yi-Ning Chang, Po-Chih Kuo

Published: 2026-04-05T14:25:04.000Z

## Abstract

This study investigates robust speech-related decoding from non-invasive MEG signals using the LibriBrain phoneme-classification benchmark from the 2025 PNPL competition. We compare residual convolutional neural networks (CNNs), an STFT-based CNN, and a CNN--Transformer hybrid, while also examining the effects of group averaging, label balancing, repeated grouping, normalization strategies, and data augmentation. Across our in-house implementations, preprocessing and data-configuration choices matter more than additional architectural complexity, among which instance normalization emerges as the most influential modification for generalization. The strongest of our own models, a CNN with group averaging, label balancing, repeated grouping, and instance normalization, achieves 60.95% F1-macro on the test split, compared with 39.53% for the plain CNN baseline. However, most of our models, without instance normalization, show substantial validation-to-test degradation, indicating that distribution shift induced by different normalization statistics is a major obstacle to generalization in our experiments. By contrast, MEGConformer maintains 64.09% F1-macro on both validation and test, and saliency-map analysis is qualitatively consistent with this contrast: weaker models exhibit more concentrated or repetitive phoneme-sensitive patterns across splits, whereas MEGConformer appears more distributed. Overall, the results suggest that improving the reliability of non-invasive phoneme decoding will likely require better handling of normalization-related distribution shift while also addressing the challenge of single-trial decoding.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Phoneme Classification on LibriBrain (val)](<https://www.sota2.com/research/sota/phoneme-classification-on-libribrain-val.md>): Phoneme Classification; LibriBrain (val); F1-macro: 71.95.

- [Phoneme Classification on LibriBrain (train)](<https://www.sota2.com/research/sota/phoneme-classification-on-libribrain-train.md>): Phoneme Classification; LibriBrain (train); F1-macro: 99.43.

- [Phoneme Classification on LibriBrain Ungrouped (test)](<https://www.sota2.com/research/sota/phoneme-classification-on-libribrain-ungrouped-test.md>): Phoneme Classification; LibriBrain Ungrouped (test); F1-macro: 4.

- [Phoneme Classification on LibriBrain (test)](<https://www.sota2.com/research/sota/phoneme-classification-on-libribrain-test.md>): Phoneme Classification; LibriBrain (test); F1-macro: 60.95.

- [Phoneme Classification on LibriBrain (holdout)](<https://www.sota2.com/research/sota/phoneme-classification-on-libribrain-holdout.md>): Phoneme Classification; LibriBrain (holdout); F1 Macro: 35.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.04129>)

[PDF](<https://arxiv.org/pdf/2604.04129.pdf>)
