Language Generation on C4 (val)
19.2OLMo PerplexityReference
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Reference2024.08 | 19.2 | — | 58.4 | |
| DGLMPrefix Guidance (w)=52024.08 | 19.4 | 85.6 | 53.9 | |
| DGLMPrefix Guidance (w)=32024.08 | 19.8 | 86.6 | 54 | |
| DGLMPrefix Guidance (w)=2.52024.08 | 20.2 | 84.8 | 54 | |
| DGLMPrefix Guidance (w)=22024.08 | 21.4 | 84.8 | 53.3 | |
| DGLMPrefix Guidance (w)=1.52024.08 | 23 | 85.6 | 52.5 | |
| GPT-2 XLParams=1.5B2026.02 | 27.1 | 0.866 | 43.3 | |
| STAR-LDMParams=0.96B, CFG (w)=22026.02 | 29.8 | 0.94 | 45.4 | |
| DGLMPrefix Guidance (w)=12024.08 | 30.1 | 84 | 50.8 | |
| GPT-2 LargeParams=0.77B2026.02 | 30.5 | 0.852 | 41.8 | |
| STAR-LDMParams=0.96B, CFG (w)=1.52026.02 | 31.1 | 0.941 | 45.4 | |
| Pythia 1.4bParams=1.4B2026.02 | 33.1 | 0.848 | 47.1 | |
| STAR-LDMParams=0.96B, CFG (w)=12026.02 | 36.1 | 0.946 | 45.3 | |
| Pythia-1bParams=1B2026.02 | 36.5 | 0.855 | 45.6 | |
| GPT-2Large2024.08 | 116.3 | 83.9 | 50.3 |