Audio-Visual Speech Recognition on LRS3 NoiseX babble noise (test)
11.2WER (SNR -5dB)Eff. Conf.
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Eff. Conf.#Par (M)=62, Fine tune data (h)=8182026.01 | 11.2 | 4.9 | 3.1 | 1.8 | 5.25 | |
| Dual-useBackbone=Whisper medium, #Par (M)=1390, Fine tune data (h)=19292026.01 | 11.27 | 3.41 | 1.9 | 1.15 | 4.43 | |
| Dual-useBackbone=Whisper small, #Par (M)=652, Fine tune data (h)=19292026.01 | 11.94 | 3.64 | 1.71 | 1.2 | 4.62 | |
| Dual-useBackbone=Whisper medium, #Par (M)=1390, Fine tune data (h)=6542026.01 | 11.94 | 3.61 | 2 | 1.43 | 4.74 | |
| Dual-useBackbone=Whisper medium, #Par (M)=1390, Fine tune data (h)=4332026.01 | 12.62 | 3.9 | 2.13 | 1.59 | 5.06 | |
| Dual-useBackbone=Whisper small, #Par (M)=652, Fine tune data (h)=6542026.01 | 12.88 | 3.94 | 2.14 | 1.44 | 5.1 | |
| Dual-useBackbone=Whisper small, #Par (M)=652, Fine tune data (h)=4332026.01 | 12.92 | 4.31 | 2.33 | 1.6 | 5.29 | |
| Llama-AVASR#Par (M)=>8000, Fine tune data (h)=17562026.01 | 16.4 | 4.2 | 2.3 | 0.9 | 5.95 | |
| MMS-Llama#Par (M)=>3000, Fine tune data (h)=17592026.01 | 42.58 | 9.7 | 3.11 | 0.88 | 14.07 |