Language Modeling on BookCorpus (test)
10.83PPLSandwich
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SandwichSandwich coefficient (k)=62019.11 | 10.83 | — | — | |
| KNN-LM2019.11 | 10.89 | — | — | |
| BaselineNumber of runs=52019.11 | 11.89 | — | — | |
| Subspace Networks (Decentralized Compressed)B/W=80Mbps, TPS=592.412025.06 | 17.63 | — | — | |
| CentralizedB/W=100Gbps, TPS=602.572025.06 | 18.35 | — | — | |
| Full-RankFLOPs (×10^9)=8.22025.12 | 28.7 | — | — | |
| DR-RLFLOPs (×10^9)=4.82025.12 | 29.8 | — | — | |
| Adaptive SVDFLOPs (×10^9)=5.32025.12 | 30.2 | — | — | |
| Fixed Low-RankFLOPs (×10^9)=4.92025.12 | 31.5 | — | — | |
| Random RankFLOPs (×10^9)=5.12025.12 | 33.1 | — | — | |
| LSTMautoregressive_components=true, dropout=none, parameters=15.0M2019.02 | 36.28 | 41.49 | — | |
| RNN-VAEautoregressive_components=true, dropout=none, parameters=23.7M, stochastic_layers=1, training_strategy=WARMUP2019.02 | 38.21 | 42.09 | 1.61 | |
| HYBRID VAEautoregressive_components=true, dropout=none, parameters=23.7M, stochastic_layers=1, training_strategy=FINETUNED2019.02 | 38.7 | 42.24 | 4.67 | |
| BIVAautoregressive_components=true, dropout=none, parameters=23.0M, stochastic_layers=7, training_strategy=FINETUNED2019.02 | 39.04 | 42.34 | 10.15 | |
| RNN-VAEautoregressive_components=true, dropout=none, parameters=23.7M, stochastic_layers=1, training_strategy=FINETUNED2019.02 | 39.26 | 42.41 | 5.13 | |
| BIVAautoregressive_components=false, dropout=none, parameters=14.0M, stochastic_layers=7, training_strategy=FINETUNED2019.02 | 108.3 | 54.13 | 15.33 | |
| DecentralizedB/W=80Mbps, TPS=36.122025.06 | 108.85 | — | — | |
| HYBRID VAEautoregressive_components=false, dropout=none, parameters=15.0M, stochastic_layers=1, training_strategy=FINETUNED2019.02 | 112.1 | 54.53 | 14.1 |