Text Continuation on WikiText-103 (test, 512-token context)
1Perplexity (PPL)Dream-B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Dream-BSize (B)=7.00, Steps=12025.09 | 1 | 1.55 | 0.005 | |
| LLaDA-ISize (B)=8.00, Steps=12025.09 | 8 | 0.53 | 0.012 | |
| LLaDA-ISize (B)=8.00, Steps=22025.09 | 8 | 0.41 | 0.014 | |
| LLaDA-ISize (B)=8.00, Steps=42025.09 | 9 | 0.4 | 0.029 | |
| LLaDA-ISize (B)=8.00, Steps=82025.09 | 9 | 0.4 | 0.027 | |
| LLaDA-ISize (B)=8.00, Steps=162025.09 | 9 | 0.43 | 0.048 | |
| Dream-ISize (B)=7.00, Steps=12025.09 | 46 | 0.58 | 0.005 | |
| TS-DFMSize=0.17B, Steps=162026.05 | 53.97 | 7.86 | 0.707 | |
| Dream-ISize (B)=7.00, Steps=162025.09 | 57 | 0.1 | 0.006 | |
| TS-DFMSize=0.17B, Steps=82026.05 | 57.11 | 7.8 | 0.339 | |
| FS-DFMSize (B)=1.30, Steps=162025.09 | 59 | 7.99 | 0.583 | |
| FS-DFMSize (B)=1.70, Steps=162025.09 | 61 | 8.06 | 0.55 | |
| Dream-ISize (B)=7.00, Steps=82025.09 | 62 | 0.13 | 0.006 | |
| Dream-ISize (B)=7.00, Steps=22025.09 | 64 | 0.35 | 0.006 | |
| Dream-ISize (B)=7.00, Steps=42025.09 | 66 | 0.2 | 0.007 | |
| FS-DFMSize (B)=0.17, Steps=162025.09 | 67 | 7.97 | 0.39 | |
| FS-DFMSize=0.17B, Steps=162026.05 | 67.42 | 7.97 | 0.39 | |
| TS-DFMSize=0.17B, Steps=42026.05 | 68.95 | 7.7 | 0.038 | |
| FS-DFMSize (B)=1.30, Steps=82025.09 | 70 | 8.01 | 0.48 | |
| FS-DFMSize (B)=1.70, Steps=82025.09 | 72 | 8.07 | 0.311 | |
| FS-DFMSize (B)=0.17, Steps=82025.09 | 75 | 7.95 | 0.27 | |
| FS-DFMSize=0.17B, Steps=82026.05 | 75.78 | 7.95 | 0.27 | |
| FS-DFMSize (B)=0.17, Steps=42025.09 | 97 | 7.89 | 0.053 | |
| FS-DFMSize=0.17B, Steps=42026.05 | 97.07 | 7.89 | 0.053 | |
| FS-DFMSize (B)=1.30, Steps=42025.09 | 99 | 7.98 | 0.12 | |
| FS-DFMSize (B)=1.70, Steps=42025.09 | 101 | 8.03 | 0.083 | |
| FS-DFMSize (B)=0.17, Steps=22025.09 | 143 | 7.85 | 0.008 | |
| FS-DFMSize (B)=1.70, Steps=22025.09 | 155 | 7.93 | 0.014 | |
| FS-DFMSize (B)=1.30, Steps=22025.09 | 169 | 7.97 | 0.013 | |
| FS-DFMSize (B)=0.17, Steps=12025.09 | 173 | 7.67 | 0.006 | |
| FS-DFMSize (B)=1.70, Steps=12025.09 | 191 | 7.67 | 0.007 | |
| FS-DFMSize (B)=1.30, Steps=12025.09 | 231 | 7.88 | 0.006 | |
| LLaDA-BSize (B)=8.00, Steps=12025.09 | 256 | 0.84 | 0.005 | |
| LLaDA-BSize (B)=8.00, Steps=22025.09 | 290 | 0.59 | 0.005 | |
| LLaDA-BSize (B)=8.00, Steps=162025.09 | 432 | 0.5 | 0.005 | |
| LLaDASize=8.0B, Steps=162026.05 | 432.65 | 0.5 | 0.005 | |
| LLaDA-BSize (B)=8.00, Steps=82025.09 | 441 | 0.42 | 0.005 | |
| LLaDASize=8.0B, Steps=82026.05 | 441.26 | 0.42 | 0.005 | |
| LLaDA-BSize (B)=8.00, Steps=42025.09 | 495 | 0.47 | 0.005 | |
| LLaDASize=8.0B, Steps=42026.05 | 495.17 | 0.47 | 0.005 | |
| Dream-BSize (B)=7.00, Steps=162025.09 | 630 | 2.31 | 0.005 | |
| DreamSize=7.0B, Steps=162026.05 | 630.3 | 2.31 | 0.005 | |
| Dream-BSize (B)=7.00, Steps=82025.09 | 739 | 1.74 | 0.006 | |
| DreamSize=7.0B, Steps=82026.05 | 739.4 | 1.74 | 0.006 | |
| Dream-BSize (B)=7.00, Steps=42025.09 | 752 | 1.53 | 0.005 | |
| DreamSize=7.0B, Steps=42026.05 | 752.11 | 1.53 | 0.005 | |
| Dream-BSize (B)=7.00, Steps=22025.09 | 785 | 1.43 | 0.005 |