Automatic Speech Recognition on LibriSpeech 960h (dev-other)
2.4WERw2v-BERT XXL
Evaluation Results
| Method | Links | |
|---|---|---|
| w2v-BERT XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=No LM2021.08 | 2.4 | |
| w2v-BERT XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=With LM2021.08 | 2.4 | |
| HuBERT X-LargeUnlabeled Data (hrs)=60k, AM Size (B)=1.0, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 2.5 | |
| w2v-BERT XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 2.6 | |
| w2v-Conformer XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=With LM2021.08 | 2.6 | |
| w2v-Conformer XXL+Unlabeled Data (hrs)=60k, AM Size (B)=1.1, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=With LM2021.08 | 2.6 | |
| w2v-BERT XLUnlabeled Data (hrs)=60k, AM Size (B)=0.6, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=No LM2021.08 | 2.6 | |
| w2v-BERT XLUnlabeled Data (hrs)=60k, AM Size (B)=0.6, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=With LM2021.08 | 2.6 | |
| w2v-BERT XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=No LM2021.08 | 2.7 | |
| wav2vec 2.0Unlabeled Data (hrs)=60k, AM Size (B)=0.3, LM Size (B)=> 0.4, Training Strategy=Pre-training + Self-training, Decoding Strategy=With LM2021.08 | 2.7 | |
| w2v-Conformer XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=No LM2021.08 | 2.7 | |
| w2v-Conformer XXL+Unlabeled Data (hrs)=60k, AM Size (B)=1.1, LM Size (B)=0.1, Training Strategy=Pre-training + Self-training, Decoding Strategy=No LM2021.08 | 2.7 | |
| w2v-BERT XLUnlabeled Data (hrs)=60k, AM Size (B)=0.6, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 2.8 | |
| w2v-BERT XLUnlabeled Data (hrs)=60k, AM Size (B)=0.6, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=No LM2021.08 | 2.9 | |
| wav2vec 2.0Unlabeled Data (hrs)=60k, AM Size (B)=0.3, LM Size (B)=> 0.4+, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 3 | |
| HuBERT LargeUnlabeled Data (hrs)=60k, AM Size (B)=0.3, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 3 | |
| w2v-Conformer XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 3 | |
| Conformer L with NSTUnlabeled Data (hrs)=60k, AM Size (B)=0.1, LM Size (B)=0.1, Training Strategy=Self-training Only, Decoding Strategy=With LM2021.08 | 3.1 | |
| wav2vec 2.0Unlabeled Data (hrs)=60k, AM Size (B)=0.3, LM Size (B)=> 0.4, Training Strategy=Pre-training + Self-training, Decoding Strategy=No LM2021.08 | 3.1 | |
| w2v-Conformer XLUnlabeled Data (hrs)=60k, AM Size (B)=0.6, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=With LM2021.08 | 3.2 | |
| w2v-Conformer XXLUnlabeled Data (hrs)=60k, AM Size (B)=1.0, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=No LM2021.08 | 3.2 | |
| Conformer L with NSTUnlabeled Data (hrs)=60k, AM Size (B)=0.1, LM Size (B)=0.1, Training Strategy=Self-training Only, Decoding Strategy=No LM2021.08 | 3.3 | |
| w2v-Conformer XLUnlabeled Data (hrs)=60k, AM Size (B)=0.6, LM Size (B)=0.1, Training Strategy=Pre-training Only, Decoding Strategy=No LM2021.08 | 3.5 | |
| HuBERT-L + Llama 8B (4bit, r=32)Train=Joint, Decode=PF-S2T2026.06 | 3.6 | |
| Whisper-LDecode=Beam2026.06 | 3.9 | |
| HuBERT-LTrain=CTC, Decode=Greedy2026.06 | 4 | |
| HuBERT-L + Llama 8B (4bit, r=32)Train=PF-S2T, Decode=PF-S2T2026.06 | 4.2 | |
| HuBERT-L + Llama 8B (4bit, r=32)Train=Joint, Decode=S2T2026.06 | 4.2 | |
| Conformer LUnlabeled Data (hrs)=N/A, AM Size (B)=0.1, LM Size (B)=0.1, Training Strategy=Trained from Scratch, Decoding Strategy=No LM2021.08 | 4.4 | |
| wav2vec 2.0Unlabeled Data (hrs)=60k, AM Size (B)=0.3, LM Size (B)=> 0.4+, Training Strategy=Pre-training Only, Decoding Strategy=No LM2021.08 | 4.5 | |
| Whisper-L + Llama 8B (4bit, r=32)Train=Joint, Decode=PF-S2T2026.06 | 4.9 | |
| HuBERT-L + Llama 8B (4bit, r=32)Train=S2T, Decode=S2T2026.06 | 4.9 | |
| Whisper-L + Llama 8B (4bit, r=32)Train=Joint, Decode=S2T2026.06 | 5 | |
| Whisper-L + Llama 8B (4bit, r=32)Train=PF-S2T, Decode=PF-S2T2026.06 | 5.5 | |
| Whisper-L + Llama 8B (4bit, r=32)Train=S2T, Decode=S2T2026.06 | 5.7 | |
| PDS-Base-16Architecture=Conformer, Group=E, L=12, dh=512, dff=2048, h=8, #Params=114M2023.05 | 6.31 | |
| Stack-4 (Baseline)Architecture=Conformer, Group=E, L=12, dh=512, dff=2048, h=8, #Params=113M2023.05 | 6.47 | |
| PDS-Base-8Architecture=Conformer, Group=E, L=12, dh=512, dff=2048, h=8, #Params=113M2023.05 | 6.54 | |
| Stack-4 (wenet)Architecture=Conformer, Group=E, L=12, dh=512, dff=2048, h=8, #Params=109M2023.05 | 6.6 | |
| PDS-Base-32Architecture=Conformer, Group=E, L=12, dh=512, dff=2048, h=8, #Params=119M2023.05 | 6.67 | |
| PDS-Base-16Architecture=Conformer, Group=D, L=12, dh=256, dff=2048, h=4, #Params=46M2023.05 | 6.97 | |
| PDS-Base-8Architecture=Conformer, Group=D, L=12, dh=256, dff=2048, h=4, #Params=46M2023.05 | 7.07 | |
| PDS-Base-32Architecture=Conformer, Group=D, L=12, dh=256, dff=2048, h=4, #Params=47M2023.05 | 7.15 | |
| Stack-4 (Baseline)Architecture=Conformer, Group=D, L=12, dh=256, dff=2048, h=4, #Params=45M2023.05 | 7.23 | |
| PDS-Base-8Architecture=Transformer, Group=B, L=12, dh=512, dff=2048, h=8, #Params=75M2023.05 | 7.46 | |
| AIPAArchitecture=Enc-CTC, Data Augmentation=SpecAugment + AIPA, Auxiliary Loss=InterCTC, Technique=COS2024.06 | 7.61 | |
| PDS-Base-16Architecture=Transformer, Group=B, L=12, dh=512, dff=2048, h=8, #Params=76M2023.05 | 7.73 | |
| PDS-Deep-32Architecture=Transformer, Group=C, L=30, dh=256, dff=2048, h=4, #Params=55M2023.05 | 7.77 | |
| PDS-Deep-16Architecture=Transformer, Group=C, L=30, dh=256, dff=2048, h=4, #Params=54M2023.05 | 7.83 | |
| PDS-Deep-8Architecture=Transformer, Group=C, L=30, dh=256, dff=2048, h=4, #Params=53M2023.05 | 7.9 | |
| PDS-Base-32Architecture=Transformer, Group=B, L=12, dh=512, dff=2048, h=8, #Params=82M2023.05 | 7.94 | |
| Stack-4 (fairseq)Architecture=Transformer, Group=B, L=12, dh=512, dff=2048, h=8, #Params=71M2023.05 | 8 | |
| Stack-4 (Baseline)Architecture=Transformer, Group=B, L=12, dh=512, dff=2048, h=8, #Params=71M2023.05 | 8.15 | |
| Stack-4 (Baseline)Architecture=Transformer, Group=C, L=30, dh=256, dff=2048, h=4, #Params=53M2023.05 | 8.51 | |
| PDS-Base-8Architecture=Transformer, Group=A, L=12, dh=256, dff=2048, h=4, #Params=30M2023.05 | 8.63 | |
| PDS-Base-16Architecture=Transformer, Group=A, L=12, dh=256, dff=2048, h=4, #Params=30M2023.05 | 8.73 | |
| Stack-4 (fairseq)Architecture=Transformer, Group=A, L=12, dh=256, dff=2048, h=4, #Params=30M2023.05 | 8.9 | |
| MELDSize=260M, Hrs=960, Beam search size=52026.05 | 9 | |
| Stack-4 (Baseline)Architecture=Transformer, Group=A, L=12, dh=256, dff=2048, h=4, #Params=30M2023.05 | 9.26 | |
| PDS-Base-32Architecture=Transformer, Group=A, L=12, dh=256, dff=2048, h=4, #Params=31M2023.05 | 9.31 | |
| BaselineArchitecture=Enc-CTC, Data Augmentation=SpecAugment, Auxiliary Loss=InterCTC2024.06 | 9.34 | |
| MELDSize=200M, Hrs=960, Beam search size=52026.05 | 9.8 | |
| dMel (ASR)Size=258M, Hrs=960, Beam search size=52026.05 | 10.3 | |
| MELD w/o SpecAugSize=200M, Hrs=960, Beam search size=52026.05 | 12.5 | |
| Codec-LMSize=200M, Hrs=960, Beam search size=52026.05 | 16.5 | |
| Codec-LM w/o codebook initSize=200M, Hrs=960, Beam search size=52026.05 | 100 |