# Neural network-based virtual microphone estimation with virtual microphone and beamformer-level multi-task loss

Source: [Neural network-based virtual microphone estimation with virtual microphone and beamformer-level multi-task loss](<https://www.sota2.com/research/paper/68fa7ffd1e209131cea9ba8a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T06:55:30.729Z

Authors: Hanako Segawa, Tsubasa Ochiai, Marc Delcroix, Tomohiro Nakatani, Rintaro Ikeshita, Shoko Araki, Takeshi Yamada, Shoji Makino

Published: 2023-11-20T08:18:16.000Z

## Abstract

Array processing performance depends on the number of microphones available. Virtual microphone estimation (VME) has been proposed to increase the number of microphone signals artificially. Neural network-based VME (NN-VME) trains an NN with a VM-level loss to predict a signal at a microphone location that is available during training but not at inference. However, this training objective may not be optimal for a specific array processing back-end, such as beamforming. An alternative approach is to use a training objective considering the array-processing back-end, such as a loss on the beamformer output. This approach may generate signals optimal for beamforming but not physically grounded. To combine the advantages of both approaches, this paper proposes a multi-task loss for NN-VME that combines both VM-level and beamformer-level losses. We evaluate the proposed multi-task NN-VME on multi-talker underdetermined conditions and show that it achieves a 33.1 % relative WER improvement compared to using only real microphones and 10.8 % compared to using a prior NN-VME approach.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Omnidirectional Speech Enhancement on Omni-SE RM: 2ch, VM: 4ch](<https://www.sota2.com/research/sota/omnidirectional-speech-enhancement-on-omni-se-rm-2ch-vm-4ch.md>): Omnidirectional Speech Enhancement; Omni-SE RM: 2ch, VM: 4ch; SI-SDR (VM-BF): 4.89.

- [Omnidirectional Speech Enhancement on Omni-SE RM: 2ch, VM: 1ch](<https://www.sota2.com/research/sota/omnidirectional-speech-enhancement-on-omni-se-rm-2ch-vm-1ch.md>): Omnidirectional Speech Enhancement; Omni-SE RM: 2ch, VM: 1ch; SI-SDR (VM-BF): 3.78.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2311.11595>)

[PDF](<https://arxiv.org/pdf/2311.11595.pdf>)