Language Modeling on Yelp (test)
4.708PPLFT(BERT(T2), Auto)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FT(BERT(T2), Auto)base_model=BERT(T2), prompt_type=Auto2022.08 | 4.708 | — | — | — | — | — | |
| FT(BERT(T2), Manual)base_model=BERT(T2), prompt_type=Manual2022.08 | 4.714 | — | — | — | — | — | |
| DCWE [temp.+social]variant=temp.+social2022.08 | 4.72 | — | — | — | — | — | |
| CWEbase_model=BERT(T2)2022.08 | 4.723 | — | — | — | — | — | |
| DCWE [temp. only]variant=temp. only2022.08 | 4.723 | — | — | — | — | — | |
| TempoBERT2022.08 | 5.516 | — | — | — | — | — | |
| HSNK (Symbols)=50, L (Walk steps)=52022.07 | 19.72 | — | — | 10.04 | — | — | |
| HSNK (Symbols)=100, L (Walk steps)=52022.07 | 19.82 | — | — | 10.2 | — | — | |
| HSNK (Symbols)=100, L (Walk steps)=202022.07 | 21.13 | — | — | 18.85 | — | — | |
| HSNK (Symbols)=50, L (Walk steps)=202022.07 | 21.6 | — | — | 24.93 | — | — | |
| Optimus Alambda=0.05, Optimized for=PPL2022.07 | 21.99 | — | — | 2.54 | — | — | |
| GPT2Fine-tuned=single epoch2022.07 | 23.4 | — | — | — | — | — | |
| Optimus Blambda=1, Optimized for=MI2022.07 | 24.59 | — | — | 9.13 | — | — | |
| iVAEMIEncoder/Decoder=one-layer LSTMs2022.07 | 36.88 | — | — | 11 | — | — | |
| LCNN-VAESize=125, pretraining=encoder pretraining using LSTM LM2017.02 | 41.1 | 359.1 | 7.6 | — | — | — | |
| MCNN-VAESize=63, pretraining=encoder pretraining using LSTM LM2017.02 | 41.8 | 360.7 | 9.1 | — | — | — | |
| LCNN-VAESize=1252017.02 | 42.3 | 361.9 | 6.4 | — | — | — | |
| LSTM-LMSize=< 32017.02 | 42.6 | 362.7 | — | — | — | — | |
| MCNN-VAESize=632017.02 | 42.8 | 363 | 6.9 | — | — | — | |
| LCNN-LMSize=1252017.02 | 43 | 363.5 | — | — | — | — | |
| SCNN-VAESize=15, pretraining=encoder pretraining using LSTM LM2017.02 | 43.1 | 363.7 | 10.3 | — | — | — | |
| VLCNN-VAESize=1872017.02 | 43.4 | 364.3 | 2.7 | — | — | — | |
| VLCNN-VAESize=187, pretraining=encoder pretraining using LSTM LM2017.02 | 43.5 | 364.7 | 2.2 | — | — | — | |
| VLCNN-LMSize=1872017.02 | 43.7 | 364.8 | — | — | — | — | |
| SCNN-VAESize=152017.02 | 43.9 | 365.6 | 9.4 | — | — | — | |
| MCNN-LMSize=632017.02 | 44.3 | 366.5 | — | — | — | — | |
| LSTM-VAESize=< 3, pretraining=encoder pretraining using LSTM LM2017.02 | 46.4 | 368.9 | 4.7 | — | — | — | |
| SCNN-LMSize=152017.02 | 46.6 | 371.2 | — | — | — | — | |
| LSTM-VAESize=< 32017.02 | 47 | 372.2 | 0.3 | — | — | — | |
| vMF-VAE (Ours)Setting=Standard2018.08 | 54 | 198 | 6.4 | — | — | — | |
| RNNLM (Ours)Setting=Standard2018.08 | 55 | 199 | — | — | — | — | |
| G-VAE (Ours)Setting=Standard2018.08 | 55 | 199 | 0.5 | — | — | — | |
| vMF-VAE (Ours)Setting=Inputless2018.08 | 134 | 242 | 48.5 | — | — | — | |
| G-VAE (Ours)Setting=Inputless2018.08 | 256 | 274 | 13.4 | — | — | — | |
| RNNLM (Ours)Setting=Inputless2018.08 | 432 | 300 | — | — | — | — | |
| CNN-VAECategory=Previous Reports2019.01 | — | 359.1 | 7.6 | — | — | — | |
| Lagging VAE2023.01 | — | — | 3.6 | 0.1 | 0.2 | -631 | |
| LIDSVAE2023.01 | — | — | 9.1 | 0.9 | 0.7 | -634.2 | |
| LSTM-LM*Objective=Standard VAE Objective2019.01 | — | 358.1 | — | — | — | — | |
| OursObjective=Standard VAE Objective2019.01 | — | 356.9 | 3.4 | 2.4 | 7.4 | — | |
| Ours + annealObjective=Modified VAE Objective, Annealing=true2019.01 | — | 355.9 | 3.8 | 2.4 | 11.3 | — | |
| SA-VAEObjective=Standard VAE Objective2019.01 | — | 357.8 | 0.3 | 0.3 | 1 | — | |
| SA-VAE2023.01 | — | — | 1.9 | 0.2 | 0.1 | -631.5 | |
| SA-VAE + annealObjective=Modified VAE Objective, Annealing=true2019.01 | — | 355.9 | 2.8 | 1.7 | 8.4 | — | |
| VAEObjective=Standard VAE Objective2019.01 | — | 358.3 | 0 | 0 | 0 | — | |
| VAE2023.01 | — | — | 0 | 0 | 0 | -635.9 | |
| VAE + annealObjective=Modified VAE Objective, Annealing=true2019.01 | — | 357.9 | 0 | 0 | 0 | — | |
| β-VAEObjective=Modified VAE Objective, beta=0.22019.01 | — | 360.7 | 11.7 | 3 | 10 | — | |
| β-VAEObjective=Modified VAE Objective, beta=0.42019.01 | — | 358.2 | 4.2 | 2 | 4.2 | — | |
| β-VAEObjective=Modified VAE Objective, beta=0.62019.01 | — | 357.9 | 0.2 | 0.1 | 3.8 | — | |
| β-VAEObjective=Modified VAE Objective, beta=0.82019.01 | — | 358.1 | 0 | 0 | 0 | — | |
| β-VAEbeta=0.22023.01 | — | — | 10 | 0.1 | 0.3 | -637.3 |