Text Continuation on One Billion Words 1K random samples (test)
21.4R-1TextLDM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TextLDMModel Type=Continuous Diffusion, Number of Parameters=768M2026.05 | 21.4 | 3.6 | 17.4 | 85 | 80 | |
| GPT-2-largeModel Type=Autoregressive, Number of Parameters=774M2026.05 | 15.8 | 2.9 | 14.6 | 84.3 | 53 | |
| GPT-2-mediumModel Type=Autoregressive, Number of Parameters=355M2026.05 | 14.8 | 2.4 | 13.8 | 84.2 | 49 | |
| GPT-2Model Type=Autoregressive, Number of Parameters=137M2026.05 | 13.4 | 1.6 | 12.3 | 83.9 | 45 | |
| Block DiffusionModel Type=Discrete Diffusion, Number of Parameters=170M, block size=162026.05 | 10.9 | 0.6 | 9.4 | 83.1 | 50 | |
| TextLDMModel Type=Continuous Diffusion, Number of Parameters=328M2026.05 | 10.8 | 0.88 | 9.8 | 83.4 | 79 | |
| TextLDMModel Type=Continuous Diffusion, Number of Parameters=114M2026.05 | 10.3 | 0.73 | 9.4 | 83.1 | 77 | |
| SSD-LMModel Type=Continuous Diffusion, Number of Parameters=355M2026.05 | 10 | 0.7 | 8.8 | 82.8 | 65 | |
| Block DiffusionModel Type=Discrete Diffusion, Number of Parameters=170M, block size=42026.05 | 9.5 | 0.8 | 8.7 | 83 | 59 | |
| Block DiffusionModel Type=Discrete Diffusion, Number of Parameters=170M, block size=82026.05 | 9 | 0.7 | 8.1 | 82.8 | 55 |