Language Modeling on Shakespeare (Perplexity and Custom Metrics)
283PerplexityNRGPT_H_FF2W
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| NRGPT_H_FF2Wlr=1e-3, min_lr=1e-4, n_layer=8, n_head=1, n_embed=128, n_params=0.2M2025.12 | 283 | 97.5 | 17.6 | 76.5 | 99.5 | |
| GPTlr=5e-3, min_lr=5e-4, n_layer=8, n_head=4, n_embed=64, n_params=0.4M2025.12 | 294 | 91.3 | 19.8 | 75.1 | 97.8 | |
| NRGPT_H_FF1lr=3e-4, min_lr=3e-5, n_layer=6, n_head=2, n_embed=512, n_params=2M2025.12 | 318 | 90.1 | 21.8 | 79.7 | 99.5 | |
| RGPT-Plr=2e-3, min_lr=2e-4, n_layer=8, n_head=1, n_embed=128, n_params=0.2M2025.12 | 410 | 88.8 | 19 | 83.8 | 100 | |
| RGPTlr=1e-3, min_lr=1e-4, n_layer=8, n_head=1, n_embed=256, n_params=0.8M2025.12 | 476 | 89.6 | 16.4 | 79.4 | 99.5 |