Language Modeling on LM1B L=128 (test)
24.53NELBO PPLEso-LMs
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Eso-LMsalpha_0=0.0625, Category=Eso-LMs (Ours)2025.06 | 24.53 | 22.39 | |
| Eso-LMsalpha_0=0.125, Category=Eso-LMs (Ours)2025.06 | 26.29 | 23.02 | |
| BD3-LMsL'=4, Category=Interpolating diffusion & AR2025.06 | 28.23 | — | |
| Eso-LMsalpha_0=0.25, Category=Eso-LMs (Ours)2025.06 | 29.23 | 24.8 | |
| BD3-LMsL'=8, Category=Interpolating diffusion & AR2025.06 | 29.83 | — | |
| BD3-LMsL'=16, Category=Interpolating diffusion & AR2025.06 | 30.6 | — | |
| MDLMCategory=Diffusion2025.06 | 31.78 | — | |
| Eso-LMsalpha_0=0.5, Category=Eso-LMs (Ours)2025.06 | 32.53 | 28.07 | |
| SEDD AbsorbCategory=Diffusion2025.06 | 32.71 | — | |
| DuoCategory=Diffusion2025.06 | 33.68 | — | |
| Eso-LMsalpha_0=1, Category=Eso-LMs (Ours)2025.06 | 36.12 | 31.65 | |
| UDLMCategory=Diffusion2025.06 | 36.71 | — | |
| SEDD UniformCategory=Diffusion2025.06 | 40.25 | — | |
| DiffusionBertCategory=Diffusion2025.06 | 63.78 | — | |
| D3PM AbsorbCategory=Diffusion2025.06 | 76.9 | — | |
| Diffusion-LMCategory=Diffusion2025.06 | 118.62 | — | |
| D3PM UniformCategory=Diffusion2025.06 | 137.9 | — | |
| + AdaLNCategory=Autoregressive (AR)2025.06 | — | 21.86 | |
| Eso-LMsalpha_0=0, Category=Eso-LMs (Ours)2025.06 | — | 21.86 | |
| TransformerCategory=Autoregressive (AR)2025.06 | — | 22.83 |