Language Modeling on OWT L=1024 (test)
20.96NELBO PPLBD3-LMs
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BD3-LMsL'=4, Category=Interpolating diffusion & AR2025.06 | 20.96 | — | |
| Eso-LMsalpha_0=0.125, Category=Eso-LMs (Ours)2025.06 | 21.92 | 20.53 | |
| BD3-LMsL'=8, Category=Interpolating diffusion & AR2025.06 | 22.04 | — | |
| BD3-LMsL'=16, Category=Interpolating diffusion & AR2025.06 | 23.57 | — | |
| Eso-LMsalpha_0=0.25, Category=Eso-LMs (Ours)2025.06 | 24.71 | 23.15 | |
| MDLMCategory=Diffusion2025.06 | 25.19 | — | |
| SEDD AbsorbCategory=Diffusion2025.06 | 26.81 | — | |
| DuoCategory=Diffusion2025.06 | 27.14 | — | |
| Eso-LMsalpha_0=0.5, Category=Eso-LMs (Ours)2025.06 | 27.94 | 26.61 | |
| Eso-LMsalpha_0=1, Category=Eso-LMs (Ours)2025.06 | 30.06 | 29.31 | |
| UDLMCategory=Diffusion2025.06 | 30.52 | — | |
| + AdaLNCategory=Autoregressive (AR)2025.06 | — | 17.78 | |
| Eso-LMsalpha_0=0, Category=Eso-LMs (Ours)2025.06 | — | 17.78 | |
| TransformerCategory=Autoregressive (AR)2025.06 | — | 17.9 |