Unconditional Text Generation on OpenWebText L=1024 (test)
14.1Generation PerplexityAR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ARNFEs=1K2026.01 | 14.1 | — | — | — | |
| Real texts2026.05 | 14.6 | 100 | 33.1 | 5.436 | |
| Duo + Ψ-samplerSteps (NFE)=4096, p=0.9, η=0.052026.05 | 18.3 | 76.5 | 19.4 | 5.2 | |
| ReMDM (rescale)Steps (NFE)=4096, p=0.92026.05 | 21.3 | 46.4 | 19.3 | 5.17 | |
| Structural Block Diffusion (Full 2-Stage)NFEs=1.5K, Tuning Data=26%2026.01 | 21.9 | — | — | — | |
| Structural Block Diffusion (Full 2-Stage)NFEs=1.2K, Tuning Data=26%2026.01 | 22.6 | — | — | — | |
| Structural Block Diffusion (Full 2-Stage)NFEs=1.1K, Tuning Data=26%2026.01 | 24.6 | — | — | — | |
| BD3-LML'=4, NFEs=1.5K2026.01 | 25 | — | — | — | |
| LDLMSteps (NFE)=40962026.05 | 25 | 21.2 | 23.3 | 5.31 | |
| BD3-LML'=4, NFEs=1K2026.01 | 25.7 | — | — | — | |
| LDLMSteps (NFE)=20482026.05 | 26.1 | 22.9 | 24.1 | 5.34 | |
| Duo + Ψ-samplerSteps (NFE)=1024, p=0.9, η=0.052026.05 | 27.08 | 61.1 | 24.2 | 5.31 | |
| Structural Block Diffusion (Stage 1 only)NFEs=1.0K, Tuning Data=26%2026.01 | 27.4 | — | — | — | |
| LDLMSteps (NFE)=10242026.05 | 27.8 | 24.6 | 25.1 | 5.37 | |
| ReMDM (rescale)Steps (NFE)=1024, p=0.92026.05 | 30.1 | 36.7 | 23.3 | 5.29 | |
| LDLMSteps (NFE)=5122026.05 | 30.2 | 28.2 | 26.7 | 5.41 | |
| BD3-LML'=8, NFEs=1K2026.01 | 30.4 | — | — | — | |
| BD3-LML'=16, NFEs=1K2026.01 | 33.4 | — | — | — | |
| LDLMSteps (NFE)=2562026.05 | 34.3 | 30.8 | 28.6 | 5.46 | |
| DuoSteps (NFE)=4096, p=0.92026.05 | 35.37 | 45.9 | 26.9 | 5.36 | |
| MDLMSteps (NFE)=4096, p=0.92026.05 | 36.4 | 21.8 | 24.4 | 5.32 | |
| SSD-LML'=25, NFEs=40K2026.01 | 37.2 | — | — | — | |
| CANDISteps (NFE)=40962026.05 | 38.8 | 13.6 | 24.2 | 5.16 | |
| LDLMSteps (NFE)=1282026.05 | 41.6 | 31.7 | 31 | 5.53 | |
| Duo + Ψ-samplerSteps (NFE)=32, p=0.9, η=0.052026.05 | 43.7 | 12 | 25.9 | 5.41 | |
| DuoSteps (NFE)=32, p=0.92026.05 | 44.22 | 13.3 | 26.1 | 5.416 | |
| MDLMNFEs=1K2026.01 | 46.8 | — | — | — | |
| CANDISteps (NFE)=322026.05 | 51.1 | 4 | 23.6 | 5.27 | |
| SEDDNFEs=1K2026.01 | 52 | — | — | — | |
| LDLMSteps (NFE)=642026.05 | 53.8 | 19.9 | 33.9 | 5.6 | |
| FLMSteps (NFE)=40962026.05 | 55.6 | 0.8 | 17.6 | 5.27 | |
| ReMDM (rescale)Steps (NFE)=32, p=0.92026.05 | 69.1 | 1.5 | 29.2 | 5.49 | |
| MDLMSteps (NFE)=32, p=0.92026.05 | 69.21 | 1.9 | 29.4 | 5.48 | |
| DuoSteps (NFE)=40962026.05 | 75.8 | 5.8 | 36.4 | 5.53 | |
| DuoSteps (NFE)=322026.05 | 95.5 | 1 | 35.9 | 5.57 | |
| MDLMSteps (NFE)=40962026.05 | 102 | 3 | 40.7 | 5.63 | |
| MDLMSteps (NFE)=10242026.05 | 104 | 3.2 | 40.9 | 5.63 | |
| LDLMSteps (NFE)=322026.05 | 132.2 | 2.3 | 42 | 5.73 | |
| MDLMSteps (NFE)=322026.05 | 197 | 0.7 | 45.4 | 5.75 | |
| FLMSteps (NFE)=322026.05 | 243.5 | 0.6 | 43.5 | 5.74 |