Unconditional Text Generation on OpenWebText (OWT) (test)
27.35Generation PerplexityTOKENDRIFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TOKENDRIFTNFE=16, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 27.35 | 5.16 | |
| DiDi-Inst.NFE=16, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 33.23 | 4.75 | |
| Di4CNFE=16, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 44.66 | 6.35 | |
| TOKENDRIFTNFE=16, Backbone=DUO, Checkpoint=13k2026.05 | 47.36 | 5.43 | |
| TOKENDRIFTNFE=8, Backbone=DUO, Checkpoint=13k2026.05 | 54.98 | 5.42 | |
| TOKENDRIFTNFE=8, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 55.04 | 5.3 | |
| SDTTNFE=16, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 65.91 | 6.42 | |
| TOKENDRIFTNFE=4, Backbone=DUO, Checkpoint=13k2026.05 | 76.17 | 5.4 | |
| DCDNFE=16, Backbone=DUO, Checkpoint=13k, Method Category=Distillation2026.05 | 81.27 | 5.54 | |
| DiDi-Inst.NFE=8, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 83.65 | 4.99 | |
| Di4CNFE=8, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 84.2 | 6.41 | |
| Cont. learningNFE=16, Backbone=DUO, Checkpoint=13k2026.05 | 114.55 | 5.58 | |
| DUONFE=16, Backbone=DUO, Checkpoint=13k2026.05 | 114.84 | 5.58 | |
| DCDNFE=8, Backbone=DUO, Checkpoint=13k, Method Category=Distillation2026.05 | 123.48 | 5.55 | |
| SDTTNFE=8, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 127.64 | 6.41 | |
| DUONFE=8, Backbone=DUO, Checkpoint=13k2026.05 | 196.94 | 5.57 | |
| Cont. learningNFE=8, Backbone=DUO, Checkpoint=13k2026.05 | 199.31 | 5.57 | |
| TOKENDRIFTNFE=4, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 208.87 | 5.41 | |
| Di4CNFE=4, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 246.89 | 6.32 | |
| DCDNFE=4, Backbone=DUO, Checkpoint=13k, Method Category=Distillation2026.05 | 299.67 | 5.51 | |
| Cont. learningNFE=16, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 337.85 | 5.78 | |
| MDLMNFE=16, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 338.81 | 5.82 | |
| DiDi-Inst.NFE=4, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 357.39 | 5.24 | |
| SDTTNFE=4, Backbone=MDLM, Checkpoint=13k, Training Budget=Not controlled (Distillation)2026.05 | 358.67 | 6.3 | |
| DUONFE=4, Backbone=DUO, Checkpoint=13k2026.05 | 508.96 | 5.54 | |
| Cont. learningNFE=4, Backbone=DUO, Checkpoint=13k2026.05 | 539.28 | 5.54 | |
| Cont. learningNFE=8, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 787.24 | 5.85 | |
| MDLMNFE=8, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 799.52 | 5.88 | |
| MDLMNFE=4, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 1,942.33 | 5.95 | |
| Cont. learningNFE=4, Backbone=MDLM, Checkpoint=13k, Training Budget=Controlled2026.05 | 1,943.95 | 5.92 |