Language Modeling on WikiText2 zero-shot
26.06PerplexityARMD
Evaluation results
| Method | Links | |
|---|---|---|
| ARMDModel Size=Medium, Training Steps=300K, Evaluation Protocol=Zero-shot2026.01 | 26.06 | |
| ARMDModel Size=Medium, Training Steps=120K, Evaluation Protocol=Zero-shot2026.01 | 27.57 | |
| RADDModel Size=Medium, Training Steps=400K, Evaluation Protocol=Zero-shot2026.01 | 30.6 | |
| SEDDModel Size=Medium, Training Steps=400K, Evaluation Protocol=Zero-shot2026.01 | 31.04 | |
| GPT-2Model Size=Medium, Evaluation Protocol=Zero-shot2026.01 | 31.8 | |
| ARMDModel Size=Small, Training Steps=400K, Evaluation Protocol=Zero-shot2026.01 | 35.64 | |
| ARMDModel Size=Small, Training Steps=180K, Evaluation Protocol=Zero-shot2026.01 | 36.25 | |
| RADDModel Size=Small, Training Steps=400K, Evaluation Protocol=Zero-shot2026.01 | 39.98 | |
| SEDDModel Size=Small, Training Steps=400K, Evaluation Protocol=Zero-shot2026.01 | 41.84 | |
| GPT-2Model Size=Small, Evaluation Protocol=Zero-shot2026.01 | 42.43 | |
| SEDD-UniformModel Size=Small, Training Steps=400K, Evaluation Protocol=Zero-shot2026.01 | 50.27 | |
| PLAIDModel Size=Small, Training Steps=600K, Evaluation Protocol=Zero-shot2026.01 | 51.8 | |
| D3PMModel Size=Small, Evaluation Protocol=Zero-shot2026.01 | 77.28 |