Language Modeling on War and Peace 32768 context length (val)
1.876LossNTK+affine
Evaluation results
| Method | Links | |
|---|---|---|
| NTK+affineTrain context length=1024, Eval context length=327682026.06 | 1.876 | |
| YaRN+affineTrain context length=1024, Eval context length=327682026.06 | 2.106 |